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Abstract 


The  Department  of  Defense  (DoD)  and  the  Air  Force  (AF) 
are  committed  to  being  national  leaders  in  protecting  and 
enhancing  the  environment  and  achieving  environmental 
compliance  (38:v;  39).  This  commitment  is  exemplified  by 
the  Air  Force  Chief  of  Staff's  goal  of  "no  notices  of 
violation  (NOVs)"  (36).  However,  the  number  of  AF  NOVs  from 
1990-1992  rose  from  103  to  178  an  increase  of  73  percent  (7; 
13;  25).  A  July  1992  AF  Inspector  General  Report  also 
concluded  that  ECAMP  findings  were  not  being  fully  utilized 
and  that  "several  commands  were  not  effectively  using 
external  Environmental  Compliance  Assessment  and  Management 
Program  (ECAMP)  reports  to  direct  corrective  actions  or 
allocate  resources"  (4:6). 

To  aid  AF  leaders  in  solving  these  problems,  and 
protecting  and  enhancing  the  environment,  this  research 
developed  a  methodology  to  standardize  and  use  ECAMP 
findings,  along  with  historical  NOV  data,  to  assist 
decision-makers  in  identifying  the  installations  that 
require  resource  allocation  to  prevent  future  environmental 
problems  and  improve  environmental  compliance.  The  ECAMP 
Effectiveness  Model  developed  in  this  research  provides  a 
quick  and  easy,  visual  indication  of  an  installation's 
relative  ECAMP  effectiveness.  The  research  indicates  that 
the  proposed  model  is  extremely  flexible;  it  can  be  easily 


X 


adjusted  to  reflect  changing  AF  or  command  goals  or 
averages;  it  can  be  used  both  retrospectively  and 
prospectively  with  either  standardized  and/or  unstandardized 
ECAMP  findings;  and  it  uses  manpower,  NOV,  and  ECAMP  report 
data  that  is  readily  available  to  decision-makers.  Based  on 
these  research  findings,  it  was  recommended  that  Air  Staff 
and  Major  Command  (MAJCOM)  environmental  leaders  review  the 
research  results  and  adopt  the  proposed  ECAMP  Effectiveness 
Model  as  an  AF  metric  to  measure  ECAMP  effectiveness. 
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DEVELOPMENT  OF  AN  EFFECTIVENESS  MODEL  OR  METRIC  FOR  THE 


AIR  FORCE  ENVIRONMENTAL  COMPLIANCE  ASSESSMENT 
AND  MANAGEMENT  PROGRAM  (ECAMP) 

I .  Introduction 


General  Issue 

The  Department  of  Defense  (DoD)  and  Air  Force  (AF)  are 
committed  to  being  national  leaders  in  protecting  and 
enhancing  the  environment  and  achieving  environmental 
compliance  (38;v;  39:1).  The  challenge  facing  DoD  and  the 
AF  is  to  achieve  compliance  with  the  multitude  of  dynamic 
and  complex  environmental  laws  and  regulations  in  the  face 
of  shrinking  budgets  and  force  reductions  (28:243). 

Although  DoD  and  the  AF  have  been  working  diligently  to  meet 
this  challenge,  a  recent  study  by  Salthouse,  Brown,  and  Oh 
indicates  that  the  number  of  notices  of  violation  (NOVs) 
issued  to  DoD  by  federal,  state  and  local  regulators 
increased  from  approximately  140  in  FY83  to  626  in  FY89 
(38:1-1).  In  addition,  the  AF  currently  has  over  180  open 
enforcement  items  (12) .  To  reverse  this  trend,  a  method  is 
needed  to  identify  installations  with  potential 
environmental  compliance  problems.  By  developing  such  a 
method,  scarce  resources  can  be  properly  allocated  to 
prevent  compliance  problems  and  protect  and  enhance  the 
environment . 
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Environmental  Auditing 

One  of  the  most  common  means  of  preventing  compliance 
problems  is  proactive  environmental  auditing  (22:1706). 

Many  businesses,  such  as  Du  Pont,  have  been  performing 
envir''  mental  audits  since  the  1970s,  but  environmental 
auda  -ing  really  gained  impetus  after  the  Environmental 
Protection  Agency  (EPA)  established  a  formal  policy  in  1986 
(35:72).  This  policy  encourages  regulated  entities  to 
develop,  implement,  and  upgrade  environmental  auditing 
programs  (24:25004).  More  specifically,  the  EPA  policy 
formally  endorses  environmental  auditing  at  federal 
facilities  (24:25004). 

As  defined  in  the  EPA  policy  statement,  an 
environmental  audit  is: 

A  systematic,  documented,  periodic  and  objective  review 
by  regulated  entities  of  facility  operations  and 
practices  related  to  meeting  environmental 
requirements.  (24:25006) 

The  primary  objective  of  environmental  audits  is  to  assess 
and  improve  environmental  compliance  (35:72) .  In  1988  the 
AF  implemented  the  EPA  policy,  by  initiating  the  AF 
Environmental  Compliance  Assessment  and  Management  Program 
(ECAMP) .  Another  key  reason  for  implementing  ECAMP  was  to 
curb  the  increasing  number  of  regulatory  NOVs  by  using 
proactive  environmental  leadership  techniques (40: 1) . 
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ECAMP 


ECAMP  is  a  comprehensive  self-evaluation  and  program 
management  system  that  is  designed  to  achieve,  maintain,  and 
monitor  environmental  compliance  at  all  AF  installations 
(18:1;  40:1).  ECAMP  utilizes  internal  and  external 
environmental  compliance  evaluations  or  audits  and 
management  action  plans  to  accomplish  these  goals  (18:1). 
Internal  evaluations  are  performed  by  installation  personnel 
under  the  direction  of  the  local  Environmental  Protection 
Committee  (EPC) ,  and  external  audits  are  conducted  by 
contractors  or  Major  Command  (MAJCOM)  personnel  not  directly 
associated  with  the  evaluated  activities  (18:5).  Internal 
evaluations  are  required  annually,  and  an  external  audit 
must  be  accomplished  at  least  once  every  three  years 
(18:3,5) . 

The  major  ECAMP  objectives  as  stated  in  AF  Regulation 
19-16  are  to: 

a.  Improve  AF  environmental  management  worldwide. 

b.  '"mprove  AF  environmental  compliance  and  compliance 
u'lanagement  in  the  U.S.  and  Possessions. 

c.  Build  supporting  financial  programs  and  budgets  for 
environmental  compliance  requirements. 

d.  Ensure  that  MAJCOMs,  installation  commanders, 
environmental  protection  committees,  environmental 
coordinators,  bioenvironmental  engineers,  and 
natural  resource  managers  environmental  programs 
are  effectively  addressing  environmental  problems. 

e.  Anticipate  and  prevent  future  environmental 
problems.  (18:3) 
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The  AF  has  been  using  ECAMP  evaluations  in  an  attempt 
to  achieve  these  objectives  since  1988;  however,  despite 
diligent  efforts,  the  number  of  NOVs  issued  remains  far 
above  the  Air  Force  Chief  of  Staff's  goal  of  "no  notices  of 
violation"  (36) .  Figure  1  illustrates  the  number  of  AF  NOVs 
from  1990  -  1992.  As  the  chart  clearly  shows,  five  years 
after  the  inception  of  ECAMP,  NOVs  continue  to  rise. 


NunawctNOW  I _ W _ I _ _ 1 _ 17t 

CALENDAR  YEAR 

MUlICBt  AP  MOMNAL  OOHMUNCK  omen 


Figure  1.  AF  Notices  of  Violation,  1990  -1992  (7;  13;  25) 

In  fact,  the  number  of  NOVs  over  this  two  year  period 
actually  increased  by  73  percent  from  103  to  178.  Although 
this  increase  could  be  due  in-pax^,  t.  the  increasing  number. 
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complexity,  and  enforcement  of  environmental  regulations,  it 
clearly  depicts  the  degree  of  effort  necessary  to  achieve 
the  Chief  of  Staff's  NOV  goal. 

In  order  to  attain  this  goal,  the  AF  must  find  a  better 
method  to  use  ECAMP  findings  to  anticipate  and  prevent 
future  environmental  problems  thereby  reducing  NOVs.  A 
July  1992  AF  Inspector  General  Report  concluded  that  ECAMP 
findings  were  not  being  fully  utilized  and  that  "several 
commands  were  not  effectively  using  the  external  ECAMP 
reports  to  direct  corrective  actions  or  allocate  resources" 
(4:6).  Ideally,  Air  Staff  and  MAJCOM  program  managers 
should  be  able  to  compare  the  number  and  type  of  ECAMP 
findings  from  various  installations  and  identify  the 
installations  that  require  resources  to  prevent  future  NOVs 
and  improve  environmental  compliance  (38:3-7). 

In  reality,  these  decisions  are  confounded  and  made 
difficult  because  bases  vary  significantly  by  size  and 
mission,  with  larger  installations  tending  to  have  more 
ECAMP  findings  and  NOVs  than  smaller  installations  (38:3-7). 
For  example,  how  can  a  decision-maker  coippare  the  number  of 
ECAMP  findings  and  NOVs  between  a  large  maintenance  depot 
and  a  much  smaller  installation  with  mobility  aircraft  and 
missiles?  To  account  for  these  differences,  ECAMP  findings 
should  be  standardized  or  weighted  to  allow  unbiased 
comparisons  between  dissimilar  organizations  (38:3-7). 
Standardizing  or  weighting  is  accomplished  by  adjusting  the 
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value  to  be  standardized  by  an  unbiased  weighting  factor 
that  accounts  for  the  variabilities  between  installations. 

If  an  unbiased  standardizing  factor  can  be  established, 

ECAMP  findings  could  be  standardized,  and  a  methodology  that 
uses  standardized  ECAMP  findings  to  identify  installations 
with  a  high  probability  of  future  compliance  problems  could 
be  developed.  Ultimately,  this  methodology  would  allow  Air 
Staff  and  MAJCOM  decision-makers  to  compare  the  number  and 
type  of  ECAMP  findings  from  various  bases  to  identify 
installations  where  resources  can  best  be  directed  to 
prevent  NOVs. 

Research  Problem 

To  attain  the  Chief  of  Staff's  goal  of  "no  notices  of 
violation"  (36)  and  ensure  that  scarce  resources  are 
properly  allocated  to  protect  and  enhance  the  environment, 
the  AF  needs  a  methodology  to  standardize  and  use  ECAMP 
findings,  along  with  historical  NOV  data,  to  assist  AF 
decision-makers  in  identifying  the  installations  that 
require  resources  to  prevent  future  environmental  problems 
and  improve  environmental  compliance. 

Research  Objectives 

The  purpose  of  this  research  was  to  develop,  test, 
evaluate,  and  revise  a  methodology  that  standardizes  ECAMP 
findings  and  uses  this  information,  along  with  historical 
NOV  data,  to  assist  AF  decision-makers  in  identifying  the 
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installations  that  require  resource  allocation  to  prevent 
future  environmental  problems  and  improve  environmental 
compliance.  Specific  research  objectives  were  to: 

1.  Demonstrate  that  environmentally-related  manpower 
determinants  can  be  used  to  standardize  ECAMP  findings 
thereby  enabling  unbiased  comparisons  between  dissimilar  AF 
installations . 

2.  Develop  a  methodology  that  uses  the  standardized 
ECAMP  findings  and  historical  NOV  data  to  assist  AF 
decision-makers  in  identifying  the  installations  that 
require  resource  allocation  to  prevent  future  environmental 
problems  and  improve  environmental  compliance. 

3.  Test,  evaluate,  and  revise  the  methodology  for 
possible  use  throughout  the  AF. 

Scope 

The  focus  of  this  research  will  be  to  develop,  test, 
evaluate,  and  refine  a  methodology  that  standardizes  and 
uses  ECAMP  findings  and  historical  NOV  data  to  assist  AF 
decision-makers  in  identifying  the  installations  that 
require  resource  allocation  to  prevent  future  environmental 
problems  and  improve  environmental  compliance.  Four 
existing  environmentally-related  AF  manpower  determinants 
will  be  objectively  evaluated  for  use  as  standardizing 
factors,  and  the  factor  that  best  allows  unbiased 
comparisons  of  ECAMP  findings  will  be  selected.  Existing 
ECAMP  findings  will  then  be  standardized  and  used  with 
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historical  NOV  data  to  develop  a  methodology  that  assists  AF 
decision-makers  in  identifying  the  installations  that 
require  resource  allocation  to  prevent  future  environmental 
problems  and  improve  environmental  compliance  for 
representative  AF  bases  within  Air  Force  Materiel,  Air 
Mobility,  and  Air  Combat  Commands.  Historical  ECAMP 
findings  will  be  obtained  from  the  respective  MAJCOMs,  and 
historical  NOV  data  will  be  provided  by  the  AF  Regional 
Compliance  Offices.  Only  external  ECAMP  findings  will  be 
used  as  these  are  the  only  audits  that  are  readily  available 
and  external  evaluations  are  generally  believed  to  be 
unbiased. 

Limitations 

Data  availability  limits  this  research  in  two  ways. 
First,  the  number  of  external  ECAMP  evaluations  will  be 
limited  as  external  audits  are  only  performed  every  three 
years.  Since  ECAMP  was  originally  implemented  in  1988, 
every  installation  should  have  had  at  least  one  external 
evaluation  to  date.  However,  this  ECAMP  data  must  be 
comparable  to  the  NOV  data.  The  amount  of  NOV  data  that  is 
available  for  comparison  is  limited  to  the  data  in  the  AF 
Regional  Compliance  Offices'  data  bases  which  were  initiated 
in  early  1990.  The  effect  of  the  limits  on  these  two  data 
sets  will  be  to  restrict  the  time-frame  of  the  study; 
however,  this  restriction  is  also  advantageous  in  that  these 
data  sets  represent  the  most  current  regulatory  period  and 
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emphasis.  Overall,  the  advantages  of  using  current  ECAMP 
and  NOV  data  may  out-weigh  the  data  availability 
limitations. 

operational  Definition  of  NOV 

For  this  research,  a  Notice  of  Violation  (NOV)  is 
defined  as  a  violation  of  environmental  laws  or  regulations 
resulting  in  enforcement  actions  by  the  EPA,  state  or  local 
regulators  and  an  entry  into  the  environmental  compliance 
data  bases  maintained  by  the  Regional  Compliance  Offices 
(RCOs)  (38:2-2).  This  operational  definition  essentially 
makes  NOVs  equivalent  to  enforcement  actions  by  applying  the 
common  and  frequently  misused  NOV  terminology.  The  reader 
should  understand  that  although  all  NOVs  are  enforcement 
actions,  not  all  enforcement  actions  are  NOVs 

Definitions 

The  definitions  provided  below  are  routinely  used  in 
ECAMP,  and  they  have  been  extracted  from  AF  Regulation  19- 
16,  "Environmental  Compliance  Assessment  and  Management 
Program"  (18:5,6) . 

Environmental  Protocol.  The  plan  and  procedures  which 
are  to  be  followed  in  conducting  an  evaluation  of  specific 
media  (air,  water,  hazardous  waste,  etc.) 

External  Evaluation.  A  single-  or  multi-  media 
evaluations  conducted  by  personnel  not  directly  associrted 
with  the  evaluated  activities. 
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Internal  Evaluation.  A  single-  or  multi-media 
assessment  conducted  by  installation  personnel  under  the 
auspices  of  the  EPC. 

Major  Finding.  A  problem  categorized  as  major  requires 
action,  but  not  necessarily  immediately. 

Minor  Finding.  Minor  findings  are  mostly 
administrative  in  nature. 

Multi-Media  Evaluation.  The  evaluation  of  two  or  more 
assessment  protocols. 

Significant  Finding.  A  problem  categorized  as 
significant  requires  immediate  action 

Thesis  Organization 

Chapter  II  of  this  study  presents  a  review  of  the 
literature  related  to  environmental  auditing,  ECAMP, 
measuring  environmental  compliance,  and  federal  facility 
compliance.  Special  emphasis  is  also  placed  on  reviewing 
the  logic  behind  using  standardized  findings,  briefly 
discussing  the  absence  of  comparable  research,  and 
describing  the  development  of  four  environmentally-related 
manpower  determinants.  Finally,  the  chapter  findings  are 
summarized,  and  conclusions  are  drawn  based  on  the 
literature  review. 

Chapter  III  describes  the  research  approach  used  to 
develop,  test,  evaluate,  and  revise  a  methodology  to  use 
standardized  ECAMP  findings  and  historical  NOV  data  to 
assist  AF  decision-makers  in  identifying  the  installations 
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that  require  resource  allocation  to  prevent  future 
environmental  problems  and  improve  environmental  compliance. 
The  chapter  begins  by  reviewing  the  research  objectives,  and 
providing  an  overview  of  the  data  collection  activities  and 
concludes  by  describing  the  general  methods  used  to  achieve 
the  specific  research  objectives. 

Detailed  research  results  and  data  analyses  are 
presented  for  the  manpower  data  analysis,  pilot  study, 
expanded  study,  and  proposed  ECAMP  Effectiveness  Model  in 
Chapter  IV.  Specifically,  this  chapter  provides  background 
information,  discusses  and  describes  the  study  results,  and 
summarizes  overall  research  results  for  each  of  these  four 
major  chapter  headings. 

Finally,  Chapter  V  reviews  the  research  problem  and 
objectives,  summarizes  and  describes  how  the  research 
results  achieved  the  three  research  objectives,  concludes 
that  the  ECAMP  Effectiveness  Model  developed  in  this 
research  provides  a  quick  and  easy,  visual  indication  of  an 
installation's  relative  ECAMP  effectiveness,  and  recommends 
that  AF  and  MAJCOM  environmental  leaders  review  the  research 
results  and  adopt  the  proposed  ECAMP  Effectiveness  Model  as 
a  metric  to  measure  ECAMP  effectiveness. 
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Literature  Review 


II . 

Overview 

This  literature  review  serves  two  distinct  purposes. 
First  to  review,  describe,  and  summarize  the  environmental 
auditing  and  ECAMP  literature  related  to  using  standardized 
external  ECAMP  findings  to  anticipate  and  prevent  future 
environmental  compliance  problems.  Second,  to  demonstrate 
that  environmentally-related  manpower  determinants  are 
indicative  of  installation  specific  environmental  compliance 
requirements.  Thus,  it  should  be  possible  to  standardize 
ECAMP  findings  using  installation  specific  environmentally- 
related  manpower  authorizations  thereby  enabling  unbiased 
comparisons  of  ECAMP  findings  from  dissimilar  AF  bases. 

The  information  is  organized  into  nine  major  sections 
with  several  sub-sections.  The  review  begins  by  discussing 
various  definitions,  essential  characteristics,  and  current 
trends  in  environmental  auditing.  Then  it  summarizes 
relevant  information  on  ECAMP  implementation,  external 
evaluations,  and  ECAMP  findings.  The  third  section  reviews 
EPA  guidance  on  measuring  environmental  compliance,  outlines 
the  difficulties  associated  with  measuring  audit  program 
effectiveness,  and  discusses  existing  ECAMP  metrics. 

Section  four  provides  background  information  on  federal 
facility  compliance  and  describes  the  potential  financial 
impacts  of  the  Federal  Facility  Compliance  Act.  Next,  the 
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absence  of  comparable  research  is  briefly  discussed,  and 
then  section  six  reviews  the  background  information  on,  and 
develops  the  logic  behind,  using  standardized  ECAMP 
findings.  Section  seven  comprises  the  longest  portion  of 
the  chapter  providing  a  detailed  review  of  the  four 
environmentally-related  manpower  determinants  that  were 
considered  as  standardizing  factors.  The  final  two  sections 
summarize  the  findings  and  draw  conclusions  based  on  the 
literature  review. 

Environmental  Auditing 

This  section  discusses  various  definitions,  essential 
characteristics,  and  current  trends  in  environmental 
auditing.  Overall,  any  methodology  or  model  that  can 
improve  environmental  audit  effectiveness  could  expand  the 
generally  accepted  characteristics  for  effective 
environmental  audits  and  start  a  new  trend  in  environmental 
auditing. 

Definitions.  As  stated  in  Chapter  I,  the  EPA  defines 
environmental  auditing  as  "a  systematic,  documented, 
periodic  and  objective  review  by  regulated  entities  of 
facility  operations  and  practices  related  to  meeting 
environmental  reguirements”  (24:25004).  Greeno  et  al  define 
environmental  auditing  as: 
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.  .  .  the  process  of  determining  whether  all  or 
selected  levels  of  an  organization  are  in  compliance 
with  regulatory  reguirements  and  internal  policies  and 
standards.  (27:3) 

Numerous  other  definitions  also  exist  and  an  ECAMP  training 
manual  developed  for  the  AF  by  Arthur  D.  Little  INC.  lists 
seven  (6: 1-3, 1-4).  In  addition  to  being  defined  in  various 
ways,  other  terms  freguently  used  to  describe  the 
environmental  auditing  process  include  environmental 
surveillance,  environmental  review,  environmental  survey, 
environmental  appraisal,  environmental  evaluation, 
environmental  guality  control,  and  environmental  assessment 
(22:1706;  34:18). 

Regardless  of  the  definitions  or  terms  used  to  describe 


the  environmental  auditing  process,  effective  environmental 

auditing  programs  generally  include  the  seven  key  elements 

discussed  in  the  EPA  Environmental  Auditina  Policv 

Statement 

.  These  elements  include: 

I. 

Explicit  top  management  support  for  environmental 
auditing  and  commitment  to  follow-up  on  audit 
findings. 

II. 

An  environmental  auditing  function  independent  of 
audited  activities. 

III. 

Adequate  team  staffing  and  auditor  training. 

IV. 

Explicit  audit  program  objectives,  scope, 
resources,  and  frequency. 

V. 

A  process  which  collects,  analyzes,  interprets 
and  documents  information  sufficient  to  achieve 
audit  objectives. 
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VI. 


A  process  which  includes  specific  procedures  to 
promptly  prepare  candid,  clear  and  appropriate 
written  reports  on  audit  findings,  corrective 
actions,  and  schedules  for  implementation. 

VII.  A  process  which  includes  quality  assurance 
procedures  to  assure  the  accuracy  and 
thoroughness  of  environmental  audits.  (24:25009) 

Although  the  EPA  published  these  elements  over  six 
years  ago,  the  current  literature  supports  their  validity. 
Hunt  lists  nine  elements  of  an  effective  environmental  audit 
program  all  of  which  can  be  found  within  the  EPA's  list 
(31:74).  Similarly,  Lear  performed  an  extensive  literature 
review,  identifying  nine  essential  characteristics  for  an 
effective  environmental  auditing  program  and  concluding  that 
the  seven  EPA  elements  included  all  of  the  nine 
characteristics  identified  in  the  review  (34:38,39).  Lear's 
essential  characteristics  are  listed  in  Table  1. 

TABLE  1 

ESSENTIAL  CHARACTERISTICS  FOR  AN  EFFECTIVE 
ENVIRONMENTAL  AUDITING  PLAN 


1.  Top  Management  Support 

2.  Commitment  to  Follow-up  on  Findings 

3 .  Independent  Auditors 

4.  Adequate  Staffing  and  Training  of  Auditors 

5.  Establishment  of  Clear  and  Specific  Program  Objectives 

6.  Clearly  Defined  Procedures 

7.  Written  Audit  Reports 

8.  Clearly  Identified  Corrective  Actions 

9.  Quality  Assurance  Procedures 


(34:39) 
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The  similarities  between  the  two  lists  are  obvious. 

However,  the  lists  seem  to  indicate  that  top  management  has 
adequate  financial  resources  to  fund  all  environmental 
compliance  requirements  identified.  In  cases  where 
requirements  exceed  resource  availability,  such  as  within 
the  AF,  managers  need  a  method  of  using  existing  audit 
findings  and  historical  noncompliance  data  to  assist  in 
ensuring  that  scarce  resources  are  properly  allocated  and 
apportioned  among  all  competing  environmental  compliance 
requirements . 

The  focus  of  this  research  is  to  develop  a  method  to 
use  standardized  external  ECAMP  findings  and  historical  NOV 
data  to  assist  AF  leaders  with  these  important  resource 
allocation  decisions.  If  successful,  this  research  will  aid 
decision-makers  in  improving  the  overall  effectiveness  and 
usefulness  of  ECAMP  thereby  assisting  the  AF  in  becoming  a 
national  leader  in  protecting  and  enhancing  the  environment. 
Essentially,  this  research  parallels  the  fifth  major  ECAMP 
objective  which  is  to  "anticipate  and  prevent  future 
environmental  problems"  (18:3).  In  general,  any  model  or 
methodology  that  can  improve  environmental  audit 
effectiveness  could  expand,  and  possibly  complete,  the  lists 
of  essential  characteristics  for  effective  environmental 
audits  and  start  a  new  trend  in  environmental  auditing. 

Trends .  Although  environmental  auditing  is  a 
relatively  new  and  evolving  discipline,  its  origin  dates 
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back  to  the  1970s  when  a  handful  of  companies  developed 
environmental  auditing  systems  as  internal  management  tools 
to  review  and  evaluate  their  own  environmental 
programs  (27:vii).  In  the  United  States,  environmental 
audits  are  not  required  by  law;  however,  the  EPA  has 
formally  endorsed  auditing  since  1986,  and  many  Fortune  500 
companies  have  been  routinely  performing  environmental 
audits  since  the  early  I980s  (11:2).  Moreover,  current 
trends  indicate  that  corporate  stockholders,  regulators,  and 
the  public  at  large  now  expect  routine  environmental  audits 
by  independent  sources  (11:2,3).  These  new  expectations  are 
compelling  companies  to  focus  more  on  th6  underlying  causes 
of  noncompliance  such  as  training,  staffing,  resource 
allocation,  and  communications  rather  than  on  superficial 
considerations  such  as  how  many  hazardous  waste  drums  were 
improperly  labelled  (11:4-7) .  The  increased  emphasis  on, 
and  visibility  of,  these  audits  is  causing  many  companies  to 
focus  not  only  on  the  compliance  issues  but  on  the 
environmental  management  systems  as  well  (11:4). 

Downs  describes  three  special-focus  audits 
(1)  environmental  complicince  audits  that  use  internal  or 
external  evaluations  to  assess  and  report  overall  company 
compliance;  (2)  environmental  management  assessments  that 
use  independent  auditors  to  evaluate  a  company's 
environmental  compliance  policies,  practices,  and  controls; 
and  (3)  criminal  liability  audits  to  assess  individual  and 
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corporate  potential  criminal  liabilities  (21:59-60).  Downs 

further  emphasizes  that: 

It  is  no  longer  sufficient  for  an  environmental 
management  system  to  identify  compliance  problems  or 
even  respond  to  specific  environmental  concerns.  To  be 
effective  in  the  1990s,  a  management  system  must 
enhance  the  corporation's  long-term  ability  to: 

Assign  responsibility  and  assure  accountability 
for  environmental  performance; 

Identify  and  assess  existing  and  potential 
compliance,  failures,  management  system 
weaknesses,  and  areas  of  potential  vulnerability; 

Communicate  such  information  to  people  who  need  to 
know ;  and 

Foster  corrective  action  by  those  responsible  for 
responding  to  and  managing  those  problems. 

(21:60) 

In  essence,  the  new  environmental  auditing  focus  on 
management  parallels  the  objectives  of  this  thesis  in  that 
they  both  seek  to  use  the  audit  results  to  assist  management 
in  anticipating  and  preventing  future  environmental 
compliance  problems. 

ECAMP 

This  section  provides  relevant  background  information 
on  ECAMP  establishment  and  implementation,  external 
evaluations,  and  findings.  In  general,  this  section 
demonstrates  the  significant  resource  demand  that  the 
environmental  compliance  requirements  identified  by  ECAMP 
findings  can  create.  Overall,  ECAMP  findings  can  and  do 
play  an  important  role  in  AF  and  installation  resource 
allocation  decisions. 


Background .  In  June  1988,  the  Air  Force  Office  of  the 
Chief  of  Staff  established  and  implemented  ECAMP  in  response 
to  reports  from  the  Government  Accounting  Office,  DoD 
Inspector  General,  AF  Audit  Agency  and  the  AF  Scientific 
Advisory  Board  that  identified  serious  environmental 
compliance  deficiencies  at  AF  facilities  (40:1).  As 
described  in  the  implementation  policy,  the  primary 
objective  of  ECAMP  is  to: 

.  .  .  establish  the  use  of  environmental  compliance 
evaluations  to  evaluate  Air  Force  compliance  with  all 
applicable.  Federal,  state,  local,  DoD,  and  USAF 
environmental  laws  and  regulations.  (40:2) 

Overall,  the  program  is  designed  to  "resolve  minor 

deficiencies  through  procedural  changes,  education,  and 

training,  and  devote  additional  resources  to  alleviate  major 

compliance  deficiencies"  (45:1).  Similarly,  the  intent  of 

this  research  is  to  use  standardized  external  ECAMP  findings 

and  historical  NOV  data  to  anticipate  compliance  problems  so 

that  additional  resources  can  be  directed  to  prevent  these 

major  compliance  deficiencies. 

External  ECAMP  Evaluations.  As  discussed  in  Chapter  I, 

external  ECAMP  evaluations  must  be  conducted  at  least  once 

every  three  years  by  personnel  not  directly  associated  with 

the  installation  (18:3).  As  a  comprehensive  evaluation, 

ECAMP  considers  any  applicable  protocol  from  the  ten  ECAMP 

protocols  described  in  Table  2.  Each  protocol  identifies 

key  federal  legislative  requirements,  typical  state  and 

local  regulations,  and  DoD  and  AF  specific  requirements  (5:1). 
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TABLE  2 


ECAMP  PROTOCOLS 


Air  Emissions 

Hazardous  Materials  Management 
Hazardous  Waste  Management 

Natural  and  Cultural  Resources 
Noise  Management 

Pesticide  Management 

POL  Management 

Solid  Waste  Management 
Special  Programs 

Water  Quality  Management 

(45:4) 

Although  these  protocols  provide  the  foundation  for  the 
audit,  an  AF  ECAMP  training  manual  developed  by  Arthur  D. 
Little  Inc.  further  delineates  the  overall  process  into 
three  distinct  sets  of  activities;  (1)  pre-assessment 
activities  such  as  scheduling,  completing  pre-visit 
questionnaires,  and  reviewing  state  and  local  regulations; 

(2)  on-site  activities  like  examining  records,  preforming 
inspections,  and  conducting  personnel  interviews;  and  (3) 
post-assessment  activities  such  as  preparing  reports  and 
conducting  follow-ups  (6:2-2;  45:3).  Similarly,  the  ECAMP 
Training  Manual  describes  the  external  ECAMP  evaluation  as  a 
nine  step  process  as  shown  in  Table  3  (45:18). 
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TABLE  3 


EXTERNAL  ECAMP  EVALUATION  PROCESS 


1.  Pre-Evaluation  Activities 

2.  On-Site  Evaluation 

3.  Outbrief  and  Preliminary  Environmental  Findings 

4.  Preliminary  Environmental  Findings  Review 

5.  Candidate  Environmental  Findings 

6.  Proposed  Action  Plan  and  Comments  on  the 
Candidate  Environmental  Findings 

7.  Draft  Final  Environmental  Evaluations  Report 

8.  Final  Environmental  Evaluation  Report 

9.  Follow-up 


(45:D-18) 


For  purposes  of  this  research,  the  important  output  of 
the  external  ECAMP  evaluation  process  is  the  external  ECAMP 
Final  Environmental  Evaluation  Report  and  the  subsequent 
findings,  or  inspection  write-ups,  detailed  therein. 

Findings.  ECAMP  findings  are  classified  as 
significant,  major,  or  minor  using  the  definitions  provided 
in  Chapter  I.  In  general,  "significant”  findings  pose 
immediate  health  or  safety  dangers  and  require  a  prompt 
response;  "major"  findings  could  result  in  NOVs,  but 
immediate  action  is  not  necessary  to  prevent  health  or 
safety  risks;  and  "minor"  findings  are  generally 
administrative  in  nature  and  are  not  as  time  sensitive 
(5:2)  . 

Overall,  significant,  major,  and  minor  ECAMP  findings 
can  and  do  play  an  important  role  in  AF  and  installation 
resource  allocation  decisions.  For  instance,  a  1991  ECAMP 
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evaluation  at  Tinker  AFB  identified  144  total  findings 
(16:80).  Within  a  few  months.  Tinker  officials  had  closed 
130  of  the  findings  at  a  cost  of  $1,253,000  and  programmed 
corrective  actions  to  close  the  other  14  findings  at  an 
estimated  cost  of  $800,000  (16:80).  For  this  one  example, 
simple  division  yields  a  closure  cost  of  over  $14,250  per 
ECAMP  finding. 

The  total  yearly  AF  expenditure  to  close-out  ECAMP 
findings  is  not  available.  However,  during  a  telephone 
interview  with  the  Director  of  Engineering  and  Services 
ECAMP  Section,  Headquarters  USAF,  Lear  learned  that  the 
ECAMP  process  identified  approximately  3,000  findings  per 
year  in  1990  and  1991  (34:3).  Combining  the  Tinker  cost 
figures  and  the  Lear  ECAMP  findings  estimate  results  in  an 
total  cost  of  approximately  $42  million  per  year  to  close¬ 
out  the  ECAMP  findings  in  1990  and  1991.  Although  this  is 
obviously  a  very  rough  estimate,  it  clearly  demonstrates  the 
significant  resource  demand  that  the  environmental 
compliance  requirements  identified  by  ECAMP  findings  can 
create. 

Measuring  Environmental  Compliance 

This  section  reviews  EPA  guidance  on  measuring 
environmental  compliance,  outlines  the  difficulties 
associated  with  measuring  audit  program  effectiveness,  and 
discusses  existing  ECAMP  metrics.  Ultimately,  this  section 
establishes  the  importance  of  measuring  ECAMP  effectiveness 
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and  the  need  for  an  improved  metric  or  new  methodology  to 
quantify  the  relationship  between  environmental  compliance 
and  environmental  audit  or  ECAMP  findings  as  suggested  by 
the  EPA. 

EPA  Guidance.  To  ensure  that  resources  are  properly 

allocated  and  apportioned  among  competing  environmental 

compliance  requirements,  such  as  equipment  maintenance, 

major  construction,  training,  or  improved  administrative 

procedures,  decision-makers  need  meaningful  environmental 

compliance  measures  (38;v).  The  1989  EPA  Publication, 

Environmental  Audit  Program  Design  Guidelines  for  Federal 

Facilities .  acknowledges  the  difficulties  associated  with 

measuring  audit  results  yet  stresses  that  some  attempt  to 

measure  results  should  be  part  of  all  audit  programs 

(23:50).  The  EPA  further  suggests  using  the  number  and 

magnitude  of  environmental  problems  identified  during  audits 

to  evaluate  overall  program  success  (23:50).  Specifically, 

the  EPA  guidance  discusses  evaluating  average  facility 

compliance  rates,  the  number  of  regulatory  compliance 

deficiencies,  and  environmental  compliance  rates  based  on 

environmental  media  and  assigned  priorities  (23:5). 

Overall,  the  EPA  indicates  that: 

A  successful  environmental  auditing  program  should 
be  reflected  in  increased  regulatory  compliance  rates 
on  a  Federal  agency-wide  basis  .  .  .  While  no  one 
measure  or  report  may  be  solely  attributable  to  the 
audit  program,  there  still  should  be  a  direct 
relationship  between  an  environmental  management 
program's  success  or  failure  and  the  effectiveness  of 
the  audit  program.  (23:50) 
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This  research  will  apply  the  EPA's  guidance  by 
comparing  and  analyzing  the  standardized  numbei  of  audit 
findings  per  installation  to  the  number  of  regulatory 
compliance  deficiencies  at  the  same  installations  over  time. 
Ultimately,  this  research  will  quantify  the  direct 
relationship  between  environmental  compliance  and  audit 
program  or  ECAMP  effectiveness. 

Measuring  Effectiveness.  Cahill  and  Kane  indicate  that 
environmental  auditing  programs  are  frequently  implemented 
without  considering  how  to  measure  program  effectiveness 
(10:VIII-6).  Although  difficult,  measuring  auditing  program 
effectiveness  is  critical  to  ensuring  environmental 
compliance.  Moreover,  truly  effective  auditing  programs 
typically  have  evaluation  and  analysis  techniques  that  use 
the  reduction  in  compliance  problems  over  time  as  a  measure 
of  success  (10:VIII-6).  As  discussed  in  Chapter  I,  the 
number  of  AF  NOVs  increased  from  103  in  1990  to  178  in  1992; 
thus,  from  a  purely  theoretical  reduction  in  compliance 
problem  perspective,  the  ECAMP  effectiveness  decreased  by  73 
percent  from  1990  to  1992  (7;  13;  25). 

Since  the  organization  and  its  executives  can  be  held 
liable  when  audits  identify  compliance  violations  and  formal 
plans  are  not  made  to  correct  the  problems,  corporate  and 
personal  liaoility  considerations  now  mandate  that  effective 
audit  programs  contain  procedures  that  assure  corrective- 
action  plans  are  in-place  and  functioning  (10;IV-29, 30) . 
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Therefore,  effective  audit  programs  must  include  systematic 
procedures  to  assure  senior  management  that  where  violations 
have  been  noted,  actions  are  being  taken  to  achieve 
compliance  (10:IV-30).  Similarly,  Air  Force  Regulation 
(AFR)  19-16,  formally  establishes  procedures  to  assure 
senior  AF  leadership  that  where  ECAMP  findings  have  been 
identified,  formal  management  action  plans  are  developed, 
and  the  appropriate  actions  are  taken  to  correct  the 
problems  (18:7)  . 

ECAMP  Metrics.  In  accordance  with  AFR  19-16,  each 
Major  Command  is  responsible  for  ensuring  effective  ECAMP 
implementation  (18:7).  As  such,  each  MAJCOM  is  free  to 
develop  its  own  ECAMP  effectiveness  measures  or  metrics. 

For  example.  Air  Combat  Command  (ACC)  has  developed  three 
ECAMP  Environmental  Quality  Performance  Measures  (EQPMs)  to: 

(1)  Determine  how  well  bases  close  findings; 

(2)  Examine  environmental  processes  for  resolving 

deficiencies;  and 

(3)  Establish  relationships  between  closure  rates  and 
resource  availability.  (9) 

Figure  2  illustrates  how  HQ  ACC  is  tracking  ECAMP  finding 
close  out  rates. 
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FY  TREND 


FISCAL  YEAR 

aOURCE:  HOAOOCEV 

Figure  2.  Sample  HQ  ACC  ECAMP  Metric  (9) 

As  the  figure  indicates,  the  primary  emphasis  in  ACC  is  on 
quick  resolution  of  ECAMP  findings. 

Similarly,  HQ  AFMC  has  established  an  ECAMP  metric 
which  measures  ECAMP  finding  closure  rates  at  90  and  180 
days  and  compares  these  values  to  the  command  goals  of  65 
and  100%  closure  respectively  (29) .  These  metrics 
effectively  assure  senior  leadership  that  appropriate 
action-plans  are  developed  and  implemented  to  correct  ECAMP 
findings.  However,  these  metrics  do  not  measure  the 
relationship  between  the  AF  environmental  management 
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program's  success  or  failure,  environmental  compliance  as 
measured  by  NOVs  received,  and  the  overall  effectiveness  of 
ECAMP  at  either  the  command  or  installation  level.  In 
general,  the  present  metrics  measure  ECAMP  effectiveness 
within  ECAMP  while  a  true  metric  should  measure  improved 
environmental  performance  external  to  ECAMP.  This  research 
develops  such  a  true  metric  to  quantify  the  relationship 
between  environmental  performance  and  ECAMP  findings  as 
suggested  by  the  EPA. 

Federal  Facility  Compliance 

This  section  provides  background  information  on  federal 
facilities  compliance,  and  it  discusses  the  potential 
financial  impacts  of  the  Federal  Facilities  Compliance  Act 
(FFCA) .  Overall,  this  section  establishes  that  the 
potential  savings  in  fines  and  penalties  that  could  be 
realized  from  this  research  are  extensive. 

Background .  The  ability  to  quantify  the  relationship 
between  ECAMP  findings  and  environmental  compliance  and 
anticipate  and  prevent  compliance  problems  will  increase  in 
importance  as  more  environmental  legislation  is  enacted. 
Hourcle  cites  congressional  testimony  by  EPA  officials  which 
indicates  that  ”63  percent  of  federal  facilities  had  one  or 
more  Class  I  violations,  compared  to  38  percent  in  the 
private  sector"  (30:376).  In  response  to  lower  compliance 
rates  for  federal  facilities  and  to  avoid  a  perceived  double 
standard  of  weaker  EPA  enforcement  of  environmental  laws 
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against  federal  facilities.  Congress  overwhelmingly  passed 

the  Federal  Facilities  Compliance  Act  of  1992  (30:359,372). 

Federal  Facilities  Compliance  Act  fFFCA) .  The  FFCA 

emphasizes  the  importance  of  AF  environmental  compliance  by 

requiring  annual  EPA  inspections  of  all  federal  facilities 

including  AF  installations  at  agency  expense,  waiving 

federal  sovereign  immunity  under  the  Resource  Conservation 

and  Recovery  Act,  and  allowing  regulators  to  assess  fines 

and  penalties  against  the  AF  for  violating  Federal,  state  or 

local  solid  and  hazardous  waste  laws  (43:1).  In  his 

article,  "EPA's  Federal  Facility  Program  -  An  Insider's 

Perspective,”  Woolf ord  indicates  that: 

The  protection  of  public  health  and  the  environment  at 
federal  facilities  is  a  high  priority  for  the 
administration,  federal  agencies,  Congress  and  the 
states.  EPA's  federal  facility  program  views  itself  as 
the  steward  for  the  public  to  ensure  that  other 
federal  agencies  comply  with  federal  environmental 
statutes,  rules,  and  regulations  to  the  same  extent  as 
required  by  the  private  sector.  (50:383) 

Ultimately,  this  means  that  the  AF  and  other  federal 

agencies  can  expect  fines  for  future  noncompliance  with  EPA 

enforced  federal  environmental  regulations.  The  maximum 

fine  for  noncompliance  is  $25,000  per  day  per  violation,  and 

past  EPA  estimates  advising  individual  bases  receiving  NOVs 

on  what  fines  would  have  been  in  the  absence  of  sovereign 

immunity  have  exceeded  $250,000  (43:1).  Thus,  when  the 

total  number  of  AF  installations  is  considered,  the 

potential  impact  on  the  AF  is  immense  (43:1).  Similarly, 

the  potential  savings  in  fines  and  penalties  that  could  be 
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realized  by  developing  a  methodology  or  model  to  use 
standardized  ECAMP  findings  and  historical  NOV  data  to 
assist  AF  decision-makers  in  determining  relative  ECAMP 
effectiveness  and  improving  environmental  compliance  as  done 
in  this  research  are  also  extensive. 

Comparable  Research 

Although  the  potential  benefits  of  using  past 
environmental  audit  findings  and  NOV  data  to  assist 
decision-makers  in  identifying  the  areas  that  require 
resources  to  prevent  future  environmental  problems  and 
improve  environmental  compliance  are  immense,  this 
literature  review  did  not  reveal  any  existing  research  in 
this  area.  The  absence  of  comparable  research  was  not 
unexpected  since  environmental  auditing  is  a  relatively  new 
and  evolving  discipline,  and  the  use  of  these  data  sources 
has  not  yet  developed  (27:vii).  Furthermore,  the  inherent 
difficulties  associated  with  assessing  audit  results,  such 
as  selecting  accurate  metrics  and  comparing  findings  from 
dissimilar  facilities,  make  predicting  future  compliance 
complex  (23:50).  Conversely,  to  reverse  the  trend  of 
increasing  NOVs  within  the  AF,  a  method  is  needed  to  use 
historical  ECAMP  findings  to  assist  decision-makers  in 
anticipating  and  preventing  future  environmental  problems 
thereby  reducing  NOVs.  As  suggested  by  Salthouse,  Brown, 
and  Oh,  the  first  step  in  comparing  environmental  compliance 
audits  is  developing  a  standardization  method  that  allows 
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unbiased  comparisons  of  installations  with  dissimilar 
environmental  compliance  requirements  (38:3-7). 

Finding  Standardization 

This  section  reviews  background  information  on,  and 
develops  the  logic  behind,  developing  a  finding 
standardization  method.  Finally,  the  section  concludes  that 
standardizing  ECAMP  findings  using  environmentally-related 
manpower  determinants  should  allow  Air  Staff  and  MAJCOM 
program  managers  to  compare  ECAMP  findings  from  dissimilar 
AF  installations. 

Background .  To  improve  efficiency  of  resource 
allocation  which  will  in-turn  prevent  noncompliance 
problems.  Air  Staff  and  MAJCOM  decision  makers  must  be  able 
to  compare  and  use  ECAMP  findings  from  various  installations 
to  identify  base  specific  projects  that  require  resources  to 
prevent  future  NOVs.  However,  comparing  findings  from 
various  size  installations  with  different  missions  in 
separate  regions  over  time  is  difficult  at  best.  Salthouse, 
Brown  and  Oh  recommend  measuring  compliance  by  using 
normalized  measures  "such  as  violations  per  source  that 
allow  unbiased  comparisons  between  dissimilar  groups  or 
organizations”  (38:3-7).  Similarly,  this  research  uses 
existing  AF  manpower  determinants  to  weight  or  standardize 
the  total  number  of  ECAMP  findings  thereby  enabling  unbiased 
comparisons  between  dissimilar  installations.  Standardizing 
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or  weighting  is  accomplished  by  adjusting  the  value  to  be 
standardized,  by  the  standardizing  factor. 

Logic.  As  described  in  the  AF  Engineering  Service 
Center  booklet,  Manpower  Management,  the  logic  behind 
manpower  determinant  development  is  that  a  required  mission 
task  demands  a  manpower  position,  and  the  basis  for  all 
manpower  positions  originates  in  the  work  requirements 
(3:5-5).  Since  installations  typically  vary  in  size  and/or 
mission,  installation  size,  mission,  and  local  regulatory 
climate  will  ultimately  dictate  the  various  environmental 
compliance  requirements  for  the  different  installations. 

Applying  similar  logic,  the  environmental  compliance 
mission  or  workload  at  different  installations  would  demand 
manpower  positions  commensurate  with  the  varying  work 
requirements  as  dictated  by  the  installation  size,  mission, 
and  local  regulatory  climate.  Since  the  manpower 
determinants  are  based  on  the  statistical  processes  of 
correlation  and  regression  analysis,  they  are  designed  to 
produce  unbiased  and  accurate  manpower  authorizations  based 
on  the  installation  specific  workload  requirements  (3:5-9). 

The  next  logical  step  is  to  extend  this  theory  and  use 
the  number  of  environmental  manpower  positions  at  a  given 
installation  to  standardize  the  number  of  ECAMP  findings  at 
that  same  installation.  In  effect,  this  will  allow  ECAMP 
findings  comparisons  between  dissimilar  installations  that 
are  as  unbiased  as  the  manpower  determinants  themselves. 


31 


Ultimately,  standardizing  ECAMP  findings  using  manpower 
authorizations  derived  from  manpower  determinants  should 
allow  unbiased  comparisons  between  dissimilar  organizations 
as  recommended  by  Salthouse,  Brown,  and  Oh  (38:3-7). 
Furthermore,  standardizing  findings  using  manpower  will 
allow  Air  Staff  and  MAJCOM  program  managers  to  compare  the 
total  number  of  ECAMP  findings  at  dissimilar  AF 
installations. 

Existing  Manpower  Determinants 

Fortunately,  several  environmentally-related  manpower 
determinants  that  could  be  used  to  standardize  ECAMP 
findings  already  exist  or  are  being  developed.  This  section 
reviews  and  discusses  the  literature  on  four  of  these 
determinants.  In  addition,  this  section  will  achieve  the 
first  research  objective  by  demonstrating  that 
environmentally-related  manpower  determinants  can  be  used  to 
standardize  ECAMP  findings  thereby  enabling  unbiased 
comparisons  between  dissimilar  AF  installations. 

Interim  Environmental  Flight  Manpower  Determinant. 

With  the  recent  AF  reorganization,  objective  wings  were 
implemented  on  1  October  1992  (46:1).  Under  this  new 
structure,  the  Civil  Engineering  squadron  oversees  the 
installation  environmental  programs  through  the  objective 
environmental  flight  (46:1).  As  described  in  the  interim 
Air  Force  Manpower  Determinant  (AFMD) ,  the  environmental 
flight  is  responsible  for: 
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.  .  .  overseeing  cleanup  of  hazardous  waste  sites, 
assisting  the  Installation  Commander  to  oversee 
compliance  with  environmental  laws,  conducting 
pollution  prevention  programs,  conducting  planning  in 
accordance  with  the  National  Environmental  Policy  Act, 
and  building  and  managing  programs  for  protection  of 
natural/cultural  resources.  (48:1) 

To  ensure  that  the  environmental  flight  is  capable  of 
achieving  its  mission,  a  team  of  experts  was  assembled  to 
assist  the  AF  Civil  Engineering  Management  Engineering  Team 
(AFCEMET)  in  developing  an  AF  Manpower  Determinant  (AFMD) 
that  "provides  the  manpower  needed  to  support  the  objective 
wing  environmental  flight  at  all  bases  with  a  flying 
mission'*  (48:1). 

The  expert  team  performed  extensive  reviews  and 
multiple  iterations  to  determine  the  environmental  processes 
and  activities  necessary  to  accomplish  the  environmental 
mission  (49:1).  Manpower  variances  were  also  developed  to 
account  for  dissimilarities  in  mission,  environmental,  and 
technological  factors  at  each  installation  (49:1). 
Additionally,  members  of  the  expert  team  and  AFCEMET 
visited  nine  bases  to  field  validate  the  core  man-hours  and 
variances  (49:1). 

The  interim  AFMD  identifies  five  core  processes, 
hundreds  of  component  activities,  total  man-hours  per  month 
and  total  man-years  for  each  core  activity,  and  eighteen 
tentatively  approved  positive  mission  variances  (15;  47) . 

The  five  core  processes  mirror  the  key  elements  in  the 
mission  statement  described  above  and  the  mission  variances 
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are  listed  in  Table  4.  The  determinant  data  is  also  used  to 


develop  a  man-hour  equation  based  on  four  workload  factors 
(48:1).  This  equation  and  the  corresponding  workload 
factors  are  shown  below: 

Man-Hour  Equation.  Y  =  576.16  +  .01077X1  +  .003191X2  + 

.08972X3  +  4.4647X4 


Workload  Factor: 

XI.  The  total  number  of  military  and  civilian 

personnel  (including  students)  authorized  on  base. 

X2.  The  total  acres  of  land  (under  facilities, 
improved,  and  semi-improved)  on  base. 

X3.  The  total  number  of  buildings  on  base. 

X4.  The  total  number  of  primary  aircraft  authorized  on 
base  including  Air  National  Guard  (ANG)  and  Air 
Force  Reserve  (AFR)  aircraft.  (48:1,2) 
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TABLE  4 


ENVIRONMENTAL  FLIGHT  VARIANCE  SUMMARY  AS  OF  9  DEC  92 


1. 

2. 

3  . 

4. 

5. 

6. 

7. 

8. 

9. 

10. 
11. 
12. 

13. 

14. 

15. 

16. 


17. 

18. 


Air  Force  Range  Operations 
Off-base  and  World-wide  Supported  Sites 
Installation  with  Treatment,  Storage,  and  Disposal 
Facility 

Host  Nation  List 
Forest  Management  Plan 
Forest  Fire  Management 
Fish  and  Wildlife  Management 

Federal/State  Threatened  and  Endangered  Species 
Management 

Grazing  and  Cropland  Management 
Coastal,  Waterways,  and  Wetlands  Management 
Historical  and  Cultural  Resource  Management 
Water,  Stormwater,  Wastewater,  and  Industrial 
Wastewater 

Compliance  Agreements  (Other  Than  DERA  Related) 

Air  Quality  Non-Attainment  Areas 
Special  Waste  Management 

Technical  Consultation  for  Private  Contractor  or 
Government  Owned-Contr actor  Operated  Environmental 
Management 

Missile  Site  Environmental  Support 
Ground  Water  Monitoring  Program 

_ (44tA-l) _ 


Work  on  the  final  AFMD  is  on-going;  however,  interim 
applications  of  the  man-hour  equation  shown  above  resulted 
in  minimum  Y-values  of  five  for  bases  outside  the  U.S.  EPA 
jurisdiction  and  six  inside  the  U.S.  EPA  jurisdiction 
(49:3).  Based  on  these  results,  installations  with  fewer 
manpower  authorizations  than  the  minimum  Y-values  were 
raised  to  the  Y-value  while  bases  with  more  authorizations 
than  the  minimums  were  fixed  at  their  assigned  strength 
(49:3).  Environmental  flight  manning  will  remain  fixed  at 
these  levels  until  FY  94  when  the  new  AFMD  will  be  applied 
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(49:1,3).  The  interim  environmental  flight  manpower 

determinant  is  very  detailed  and  attempts  to  consider  the 

multitude  of  factors  that  determine  environmental  flight 

workload  requirements.  There  is  also  a  State  Regulatory 

Impact  Factor  that  is  used  to  weight  workload  factor  X2 

(acres)  based  on  the  "Volume  of  Regulations  and  Severity  of 

Enforcement”  (15).  However,  the  interim  AFMD  does 

acknowledge  difficulties  associated  with  applying  the 

traditional  least  squares  regression  procedures  to  core 

environmental  flight  manpower  requirements.  In  a  section 

entitled  "Practical  Considerations"  the  interim  AFMD  states: 

Actions  required  to  maintain  the  installation  in  a 
state  of  compliance  originate  in  Public  Law  and  are  not 
discretionary.  The  factors  truly  predicting  manpower 
requirements  are  contained  in  the  millions  of  pages  of 
regulations  with  which  each  installation  must  comply  to 
avoid  pollution  of  the  environment,  avoid  willful  and 
knowing  endangerment  of  Flight  and  base  staff  members, 
and  avoid  potential  civil  and/or  criminal  actions  for 
acts  of  both  commission  and  omission.  Environmental 
law  does  not  require  "intent”  to  violate  a  statute  as  a 
minimum  threshold  to  be  prosecuted.  Knowledge  that  one 
is  breaking  the  law  is  not  always  readily  discernable 
considering  the  explosion  of  environmental  regulations 
with  over  55,000  pages  of  new  regulations  published 
since  1982.  (49:5) 


Although  the  concerns  described  above  are  valid,  the 
interim  environmental  flight  manpower  determinant  is  being 
applied  in  all  objective  wings,  and  it  considers  numerous 
factors  including  installation  size,  mission,  and  local 
regulatory  climate  in  determining  environmental  flight 
manpower  authorizations.  Thus,  using  the  installation 
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specific  manpower  authorizations  derived  with  this 
determinant  to  standardize  installation  ECAMP  findings 
should  allow  unbiased  comparisons  between  dissimilar 
installations  with  objective  wings. 

Environmental  Management  Manpower  Determinant .  At  the 
present  time,  the  interim  environmental  flight  manpower 
determinant  does  not  apply  to  former  Air  Force  Logistics 
Command  (AFLC)  installations  (37)  .  Instead,  an  indepth 
study  of  the  nan-hour  requirements  necessary  to  accomplish 
the  workload  of  the  former  AFLC  (now  Air  Force  Materiel 
Command)  Directorate  of  Environmental  Management  functions 
published  on  17  January  1992  is  being  used  (19:1-1).  This 
study  used  operational  audit  techniques  to  collect  workload 
data  for  all  seven  Environmental  Management  (EM) 
Directorates  (19:1-1).  This  data  revealed  17  workload 
categories  and  hundreds  of  sub-tasks  (19:1-1).  These 
categories  are  similar  to  the  core  processes,  and  the  sub¬ 
tasks  parallel  the  component  activities  in  the  interim 
environmental  flight  manpower  determinant. 

Statistical  analyses  were  performed  on  the  data,  and 
regression  models  were  used  to  develop  a  modular  equation 
application  matrix  (19:A4-2).  This  matrix  includes 
regression  equations  for  six  of  the  seventeen  workload 
categories,  and  these  equations  use  the  eleven  workload 
factors  shown  in  Table  5  (19:1-1). 
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TABLE  5 


ENVIRONMENTAL  MANAGEMENT  DIRECTORATE  WORKLOAD  FACTORS 


XI.  DD  Form  1348s  Processed 

X2.  AF  Form  813s  (Request  for  Environmental  Impact 
Assessment)  Reviewed 

X3.  Environmental  Assessments  Completed 
X4.  Construction  Projects  Reviewed 
X5.  Permitted  Outfalls  Managed 

X6.  Constituents  on  Industrial  Waste  Treatment  Plant 
Permit 

X7.  National  Priorities  List  (NPL)  Sites  Managed 

X8.  Non-NPL  Sites  Managed 

X9.  Hazardous  Materials  Spills 

XIO.  Asbestos  Air  Samples  Taken 

XI I.  Asbestos  Contract  Projects 


(19:1-1) 


Although  the  six  equations  and  eleven  workload  factors 
identified  in  the  EM  manpower  determinant  are  vastly 
different  from  the  single  regression  equation  and  five 
workload  factors  identified  in  the  environmental  flight 
manpower  determinant,  both  procedures  systematically  apply 
statistical  methods  to  determine  manpower  requirements  based 
on  the  varying  levels  of  environmental  workload  at 
dissimilar  installations.  Overall,  the  EM  determinant 
accounts  for  the  differences  between  installations  by 
increasing  both  the  number  of  equations  and  workload  factors 
while  the  environmental  flight  determinant  accounts  for 
these  differences  by  using  simpler  equations  and  fewer 
workload  factors  but  allowing  complex  installation  specific 
variances.  In  essence,  both  methods  are  of  comparable 
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complexity,  and  they  attempt  to  achieve  the  same  goal  for 
installations  in  different  commands. 

In  summary,  the  EM  manpower  determinant  is  being 
applied  at  all  AFMC  installations  with  EM  Directorates,  and 
it  considers  numerous  workload  factors  that  are  influenced 
by  installation  size,  mission,  and  local  regulatory 
enforcement.  Thus,  using  the  installation  specific  manpower 
authorizations  derived  using  this  determinant  to  standardize 
installation  ECAMP  findings  should  allow  unbiased 
comparisons  between  different  AFMC  installations  wich  EM 
functions. 

Bioenvironmental  Engineering  Manpower  Determinant. 

The  Bioenvironmental  Engineering  (BEE)  manpowei 
determinant  is  another  existing  environmentally-related 
manpower  determinant  that  applies  to  all  installations 
authorized  functional  account  code  (FAC)  5311  except  AFMC 
bases (17:1).  Unlike  the  recently  developed  environmental 
flight  and  EM  manpower  determinants  the  BEE  AFMD  has  been 
used  since  the  early  1980s,  and  it  has  gone  through  two 
major  revisions.  The  current  June  1991  AFMD,  which 
supersedes  the  July  1985  version,  is  based  on  an  operational 
audit  of  historical  records  and  technical  estimation 
techniques  (17:1).  In  contrast,  the  EM  manpower  determinant 
study  concluded  that: 

Due  to  the  relative  young  age  of  the  Environmental 
Management  organization,  the  lead  team  felt  it  prudent 
to  delay  total  development  of  the  management  decision 
package  until  at  least  the  next  measurement.  Only  6  of 
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17  functional  categories  of  work  were  performed  in  a 
similar  enough  manner  to  produce  an  acceptable 
correlation  coefficient.  As  the  organization  matures, 
and  more  measurement  data  is  collected,  the  management 
decision  package  can  be  developed.  (19:1-1) 

In  effect,  the  BEE  manpower  determinant  may  be  a  better 

standardizing  factor  since  it  has  matured  over  time  with  the 

developing  environmental  legislation.  In  contrast, 

environmental  flight  and  EM  manpower  authorizations  have  had 

a  tendency  to  increase  significantly  at  different  times  in 

response  to  local  politics,  environmental  problems,  and 

organizational  or  policy  changes.  For  instance,  Brunner 

indicates  that: 

.  .  .  during  the  mid-1980s,  it  became  clear  to 
McClellan's  management  that  we  had  to  change  the  way  we 
did  business.  Base  environmental  resources  had  to  be 
consolidated  and  expanded  in  order  to  meet  the  growing 
environmental  problems  that  the  base  faced.  (8:186) 

In  response  to  these  factors,  McClellan's  dedicated 

environmental  positions  grew  almost  immediately  from  five  or 

fewer  people  to  thirty-one  (8:187).  By  April  92,  the  number 

was  60,  and  as  of  February  93  there  were  84  dedicated 

environmental  positions  (8:187;  37).  Similarly,  several 

installations  with  objective  environmental  flights  saw  their 

manpower  authorizations  immediately  raised  as  a  result  of 

the  policy  decision  to  increase  all  objective  flight 

manpower  levels  to  the  core  values  developed  under  the 

interim  environmental  flight  manpower  determinant  (49:3). 

Like  the  EM  and  environmental  flight  AFMDs,  the  BEE 
AFMD  is  designed  to  quantify  the  manpower  requirements 
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necessary  to  accomplish  the  tasks  described  in  the  work 
center  description  for  the  varying  levels  of  workload  at 
dissimilar  installations.  The  similarities  between  the 
tasks  or  activities  in  the  work  center  descriptions  for  the 
EM  and  BEE  manpower  determinants  are  illustrated  in  Tables  6 
and  7.  Table  6  lists  the  17  major  headings  from  the  EM  work 
center  description,  and  Table  7  lists  similar  major  headings 
and  subheadings  from  the  BEE  work  center  description. 


TABLE  6 

EM  WORK  CENTER  DESCRIPTIONS 


1.  Hazardous  Waste  Program 

2.  Hazardous  Materials  Program 

3.  Natural  Resources 

4.  Cultural  Resources 

5.  Environmental  Compliance  Assessment  and  Management 
Program  (ECAMP) 

6.  Environmental  Impact  Analysis  Process  (EIAP) 

7 .  Resource  Management 

8.  Air  Quality  Program 

9.  Water  Quality  Program 

10.  Restoration  Program 

11.  Storage  Tank  Program 

12.  Spill/Response  and  Planning 

13.  Manage  Asbestos  Program 

14 .  PCB  Program 

15.  Solid  Waste  Management  Program 

16.  Project  Management 

17 .  Hazardous  Waste  Minimization  Program 


TABLE  7 


BEE  WORK  CENTER  DESCRIPTIONS 


1.  Construction,  Process  Order,  and  Equipment  Design 
Review 

2.  Issue  Exception  (lEX)  Coding  Program  for  Hazardous  or 
Toxic  Substances 

3.  Workplace  Environmental  Evaluation 

3. 2. 3. 8  Collects  Pollution  Control  Data 

3. 2. 3. 9  Updates  Hazardous  Material  Data 

4 .  Community  Environment 

4.1  Assesses  Potable  Water  Quality 

4. 1.1. 3  Provides  Notification  of  Noncompliance 

4.4  Environmental  Compliance  Assessment  and 

Management  Program  (ECAMP) 

5.  Water  and  Air  Pollution  Monitoring 

5.2  Performs  National  Pollution  Discharge 
Elimination  System  (NPDES)  Monitoring 

5.3  Performs  Annual  Air  Pollution  Emission 
Inventory 

5.4.5  Conducts  Visual  Emission  Survey 

5.5  Assists  Civil  Engineering  With  State  or  Local 
License  or  Permit  Request 

5.6  Maintains  Pollution  Data 

5.7  Participates  in  Installation  Restoration 
Program 

9.  Fuel  Tank  Cleaning  Monitoring  and  Inspection 
11.  Supports  On-Base  Functions  Provided  By  Maintenance  and 
Services  Contracts 

13.  Resource  Conservation  and  Recovery  Act  (RCRA) 

Monitoring  and  Training 

13.3  Conducts  Compliance  Survey 

13.5  Performs  Waste  Stream  Characterization 

16.  Hazard  Communication  (HAZCOM)  Program 

16.1.3  Conducts  Chemical  Inventory 

16.2  Researches  and  Updates  Hazardous  Material 

Data 

_ (17) _ 


Although  the  two  tables  are  not  identical,  close  inspection 
reveals  definite  similarities.  For  instance,  items  8  and  9 
in  Table  6  identify  air  and  water  quality  programs  as 
primary  EM  work  center  activities  while  item  5  in  Table  7 
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lists  water  and  air  pollution  monitoring  as  a  primary  BEE 
work  center  tasking  and  then  goes  on  to  specify  numerous  air 
and  water  pollution  monitoring  sub-taskings .  The  sub¬ 
tasking  levels  in  the  BEE  work  center  description  may  also 
indicate  that  the  BEE  manpower  standard  has  evolved  over 
time  to  include  more  detailed  environmentally-related 
workload  factors  then  the  relatively  young  EM  manpower 
determinant. 

The  BEE  AFMD  is  also  similar  to  the  interim 
environmental  flight  manpower  determinant  in  that  it  is 
based  on  man-hour  equations  and  workload  factors  (17:1). 
However,  the  BEE  manpower  determinant  is  based  on  two  man¬ 
hour  equations  and  two  workload  factors  whereas  the  interim 
environmental  flight  AFMD  is  based  on  one  equation  with  four 
workload  factors  (17:1;  48:1,2).  The  BEE  man-hour  equations 
and  workload  factors  are  shown  below. 

Man-hour  Equations. 

(1)  Y1  =  441.2  +  0.1519X1  +  6.890X2  for  locations 

which  earn  at  least  803.603  man-hours  after 
aoplying  Equation  Yl. 

(2)  Y2  =  332.8  +  0.1706X1  =  1.324X2  for  locations 
which  earn  less  than  803.603  man-hours  after 
applying  Equation  Yl. 

Workload  Factors: 

(1)  XI.  Selected  authorized  military  and 
civilian  population. 

(2)  X2.  Air  National  Guard,  Air  Force  Reserve, 
Area  Radar  Sites,  and  other  DoD  Installations 
Supported  by  an  attending  Bioenvironmental 
Engineer.  (17:1) 
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Comparing  the  BEE  workload  factors  with  the  interim 
environmental  flight  workload  factors  discussed  earlier 
reveals  that  both  groups  of  variables  consider  the  number  of 
authorized  personnel  and  Air  National  Guard  and  Air  Force 
Reserve  activities.  Furthermore,  the  workload  factors  in 
the  July  1985  BEE  AFMD  were  almost  identical  to  those 
currently  proposed  for  environmental  flight.  The  1985  BEE 
workload  factors  included  values  for  selected  military  and 
civilian  Air  Force  Specialty  Codes  (AFSCs) ,  total  acreage, 
total  number  of  assigned  aircraft,  and  the  number  of 
supported  installations  (17:3). 

In  general,  the  BEE  AFMD  is  another  existing, 
statistically  based  manpower  determinant  that  applies  to  all 
AF  installations  except  AFMC  bases.  Like  the  EM  and  interim 
environmental  flight  AFMDs  discussed  earlier,  it  is  designed 
to  consider  installation  size  and  mission  variabilities. 
Furthermore,  the  BEE  workload  factors,  workload 
descriptions,  and  environmental  taskings  are  also  similar. 
Moreover,  the  BEE  AFMD  may  be  a  better  standardizing  factor 
since  it  has  a  longer  history  and  has  matured  over  time  with 
the  evolving  environmental  legislation.  In  essence,  using 
the  BEE  manpower  authorizations  to  standardize  the 
corresponding  installation  ECAMP  findings  should  allow 
unbiased  comparisons  for  all  AF  bases  except  AFMC 
installations. 
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Just  as  separate 


manpower  determinants  were  developed  for  the  EM  functions  at 
AFMC  installations,  separate  determinants  were  also 
developed  for  Bioenvironmental  Engineering  sections  at  these 
large  and  complex  installations.  Like  the  BEE  manpower 
determinant  just  discussed,  the  AFMC  BEE  manpower 
determinant  has  been  evolving  with  time.  The  most  recent 
functional  review  was  initiated  in  July  1991,  the  workload 
measurement  period  was  from  1  December  1989  to  31  November 
1991,  and  the  final  report  was  completed  in  October  1992 
(2:1-2).  Overall,  this  study  was  similar  to  those 
previously  discussed;  however,  the  findings,  data  analysis, 
and  determinant  development  provide  additional  unique 
insights. 

The  work  center  descriptions  developed  in  this  study 
closely  parallel  the  descriptions  used  in  the  BEE  AFMD  for 
non-AFMC  installations  (key  portions  of  which  were 
previously  identified  in  Table  7) ;  however,  nine  new 
categories  of  work  were  identified  (2:3-2).  These 
categories  support  the  covariation  between  BEE  and 
environmental  workloads,  and  they  are  listed  in  Table  8. 
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TABLE  8 


NEW  BIOENVIRONMENTAL  ENGINEERING  WORKLOAD  CATEGORIES 


18 . 1  Manages  Data 

18.1.1.4  Prepares  Environmental  Monitoring  data 

19.  Workplace  Biomonitoring 

20.  Biological  Waste  Disposal  Program  Monitoring  to 
Assess  Compliance  with  Federal,  State  and  Local 
Regulations 

22  Hazardous  Waste  Operation  and  Emergency  Response 
Medical  Support  Program 

23  Indoor  Air  Quality  (lAQ)  Program 

24  Occupational  Exposure  to  Hazardous  Chemicals 

in  Laboratories 

25  Hazardous  Identification  and  Abatement  of  Lead-Based 

Products  in  Housing  and  Public  Facilities 

26  Technical  Order  Review 

26.3.2  Reviews  Toxicity  Resource  Recovery, 

Hazardous  Waste  (Minimization, 
Substitution,  Disposal) ,  Physical, 
Biological  Hazards  or  Other  Related 
Topics 

27  Installation  Restoration  Program  (IRP) 

_ (2;3-2) _ 


During  this  study,  nine  potential  workload  factors  were 

reviewed  including  total  authoiized  personnel,  installation 

acreage,  and  total  square  footage  of  base  facilities 

(2:3-3).  However,  due  to  extreme  workload  variation  from 

installation  to  installation,  it  was  difficult  to  develop 

one  or  two  standardized  manpower  equations  (2:3-3). 

Instead,  separate  regression  equations  were  derived  for  each 

installation  using  the  total  base  populace  as  the  X-value,  a 

constant  slope  or  b-value,  and  an  installation  unique 

a-value  (2:3-3).  The  study  also  concluded  that: 

Things  such  as  state  and  local  laws  and  regulations; 
base  and  local  community  interest  items;  directives 
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from  MAJCON  level  and  higher;  and  the  amounts  of 
hazardous  material  used  and  stored  on  the  base  all 
determine  what  is  or  is  not  at  BEE  priority.  (2:3-3) 

Ultimately,  the  AFMC  BEE  functional  review  supports  the 

need  to  consider  installation  size,  mission,  and  local  laws 

and  regulations  when  selecting  a  standardization  factor. 

Additionally,  using  the  manpower  authorizations  derived  from 

the  installation  specific  manpower  regression  equations  to 

standardize  ECAMP  findings  should  allow  unbiased  comparisons 

between  dissimilar  installations  within  AFMC. 

Summary 

Environmental  auditing  has  many  definitions  and 
numerous  terms  are  used  to  describe  the  process;  however, 
the  primary  objective  of  environmental  auditing  is  to  assess 
and  improve  environmental  compliance.  Although 
environmental  auditing  is  a  relatively  new  and  evolving 
discipline,  the  literature  indicates  that  the  seven  key 
elements  discussed  in  the  1986  EPA  Environmental  Auditing 
Policy  Statement  remain  valid  indicators  of  effective 
environmental  auditing  programs. 

Moreover,  current  trends  indicate  that  the  American 
public  now  expects  routine  environmental  audits  by 
independent  sources.  These  expectations  are  causing 
companies  to  focus  audits  on  environmental  management 
systems  and  their  abilities  to  identify  and  assess  both 


existing  and  potential  environmental  compliance  problems 
In  1988,  the  AF  established  and  implemented  an 


environmental  auditing  program  known  as  ECAMP.  Like  other 
auditing  programs,  ECAMP  uses  internal  and  external 
assessments  to  evaluate  AF  environmental  compliance. 
Subsequently,  noncompliance  findings  frequently  have 
significant  impacts  on  installation  resource  allocation 
decisions.  To  ensure  that  resources  are  properly  allocated 
and  apportioned  among  competing  environmental  compliance 
requirements,  AF  decision-makers  need  meaningful 
environmental  compliance  measures.  Furthermore,  comparing 
ECAMP  results  and  measuring  overall  program  effectiveness  is 
critical  to  ensuring  environmental  compliance.  However, 
quantifying  the  relationship  between  environmental 
compliance  and  ECAMP  findings  is  difficult,  and  current 
ECAMP  metrics  focus  on  expeditious  finding  resolution 
without  systematically  considering  overall  program 
effectiveness  or  assessing  future  noncompliance 
probabilities . 

Ideally,  Air  Staff  and  MAJCOM  decision-makers  should  be 
able  to  compare  the  total  number  and  type  of  ECAMP  findings 
from  various  installations  and  identify  the  installations 
that  require  resources  to  prevent  future  environmental 
problems  and  improve  environmental  compliance.  However, 
before  ECAMP  findings  can  be  compared,  these  findings 
probably  need  to  be  weighted  or  standardized  to  account  for 
dissimilarities  between  installations. 

Ultimately,  installation  specific  environmental 
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compliance  requirements  are  a  function  of  installation  size, 
mission,  and  local  regulatory  enforcement.  Similarly,  ECAMP 
findings  are  a  function  of  the  corresponding  installation 
specific  environmental  compliance  requirements.  Thus,  there 
should  be  a  direct  relationship  between  the  number  of  ECAMP 
findings  and  environmental  compliance. 

To  quantify  this  relationship,  and  compare  ECAMP 
findings  between  dissimilar  installations,  a  weighting  or 
standardizing  factor  that  accounts  for  installation  specific 
size,  mission,  and  environmental  compliance  requirements 
would  be  useful.  Fortunately,  several  environmentally- 
related  manpower  determinants  that  are  statistically  derived 
and  specifically  designed  to  produce  unbiased  manpower 
authorizations  based  on  installation  workload  requirements 
already  exist. 

The  literature  describing  the  development  of  the 
Interim  Environmental  Flight,  AFMC  Environmental  Management, 
Bioenvironmental  Engineering,  and  AFMC  Bioenvironmental 
Engineering  manpower  determinants  indicates  that  all  four 
developmental  procedures  systematically  apply  observational, 
data  collection,  and  statistical  methods  to  determine 
specific  manpower  requirements  based  on  the  varying  levels 
of  environmental  workload  at  dissimilar  installations. 
Ultimately,  each  determinant  makes  a  comprehensive  effort  to 
consider  all  relevant  workload  requirements  as  dictated  by 
installation  size,  mission,  and  local  regulatory 
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enforcement.  In  essence,  using  each  determinant  to 
standardize  ECAMP  findings  for  the  respective  installations 
where  they  are  currently  being  applied  should  allow  unbiased 
comparisons  of  ECAMP  findings  from  dissimilar  installations 
using  the  same  manpower  determinant.  However,  the  BEE 
manpower  determinant  may  be  a  better  standardizing  factor 
since  it  has  matured  over  time  with  the  developing 
environmental  legislation.  Furthermore,  comparing  and 
statistically  analyzing  the  standardized  findings  using  the 
respective  BEE  and  environmental  AFMDs  may  prove  valuable. 

Conclusion 

The  literature  review  clearly  demonstrates  that  the 
increasing  number  of  AF  NOVs  is  inconsistent  with  AF 
environmental  compliance  goals,  ECAMP  objectives,  and 
successful  environmental  auditing  program  characteristics. 
Moreover,  the  present  ECAMP  metrics  measure  program 
effectiveness  within  ECAMP  while  a  true  metric  should 
measure  performance  external  to  the  program  through  improved 
environmental  compliance.  To  improve  compliance  and  become 
a  national  leader  in  protecting  and  enhancing  the 
environment,  the  AF  needs  a  metric  relating  standardized 
ECAMP  findings  and  historical  NOV  data  tp  assist  decision¬ 
makers  in  identifying  the  installations  that  require 
resources  to  prevent  future  environmental  problems  and 
improve  environmental  compliance.  Thus,  research  into 
developing  such  a  metric  or  model  is  needed.  If  successful. 
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this  research  could  assist  AF  decision-makers  in  properly 
allocating  and  apportioning  scarce  resources  among 
installations  with  competing  environmental  compliance 
requirements.  In  addition,  the  development  of  such  a  metric 
or  model  could  assist  decision-makers  in  improving  ECAMP 
effectiveness  thereby  reducing  NOVs  and  preventing  fines  and 
penalties  pursuant  to  the  Federal  Facilities  Compliance  Act. 


Ill .  Methodology 


Overview 

This  chapter  describes  the  methodology  that  was  used  to 
investigate  the  three  research  objectives  outlined  in 
Chapter  I : 

1.  Demonstrate  that  environmentally-related  manpower 
determinants  can  be  used  to  standardize  ECAMP  findings 
thereby  enabling  unbiased  comparisons  between  dissimilar  AF 
installations. 

2.  Develop  a  methodology  that  uses  the  standardized 
ECAMP  findings  and  historical  NOV  data  to  assist  AF 
decision-makers  in  identifying  the  installations  that 
require  resource  allocation  to  prevent  future  environmental 
problems  and  improve  environmental  compliance. 

3.  Test,  evaluate,  and  revise  the  methodology  for 
possible  use  throughout  the  AF. 

Overall,  this  research  addresses  two  problems.  First, 
the  validity  of  using  existing  environmentally-related 
manpower  determinants  as  standardizing  factors  was 
subjectively  established  in  the  literature  review.  Next,  a 
methodology  using  standardized  ECAMP  findings  and  historical 
NOV  data  to  assist  AF  decision-makers  in  identifying 
installations  that  require  resources  to  prevent  future 
environmental  problems  and  improve  environmental  compliance 
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was  developed,  tested,  and  evaluated.  Finally,  the 
methodology  was  revised  and  an  ECAMP  effectiveness  model  was 
proposed,  tested  and  evaluated.  Based  on  these  results,  it 
was  recommended  that  Air  Staff  and  MAJCOM  environmental 
leaders  review  the  research  and  consider  using  the  proposed 
ECAMP  Effectiveness  Model  as  a  metric  to  measure  ECAMP 
effectiveness  and  improve  environmental  compliance. 

Data  Collection 

This  section  describes  and  discusses  the  various  data 
collection  activities  that  occurred  during  this  research. 

Due  to  the  sensitive  nature  of  NOV  and  ECAMP  information, 
MAJCOMs  requested  that  the  identity  of  the  individual  bases 
involved  in  the  study  not  be  disclosed.  To  maintain  this 
confidentiality,  code  letters  and  numbers  were  assigned  to 
each  base  in  the  data  set.  The  actual  names  of  the 
installations  were  recorded  by  the  researcher  but  do  not 
appear  in  the  thesis. 

Manpower  Authorization  Data.  The  BEE  manpower 
authorization  data  used  for  the  standardization  process  was 
acquired  from  the  MAJCOM  BEE  offices  at  AFMC,  AMC,  and  ACC. 
Similarly,  interim  environmental  flight  and  environmental 
management  manpower  authorization  figures  were  obtained  from 
the  environmental  offices  at  the  respective  commands. 

ECAMP  Data.  The  external  ECAMP  data  analyzed  in  this 
research  was  also  acquired  from  the  MAJCOM  environmental 
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offices.  Fifty-nine  bases  were  identified  as  potential 
candidates  for  study;  thirty-four,  thirteen,  and  twelve  from 
ACC,  AMC,  and  AFMC  respectively.  Initial  plans  to  randomly 
select  the  bases  for  the  study  were  changed  based  on 
conversations  with  the  individual  MAJCOM  ECAMP  managers. 
These  conversations  revealed  that  external  ECAMP  reports 
prior  to  1991  were  not  readily  available  (9;  29;  42) .  Thus, 
the  ACC  and  AMC  program  managers  were  asked  to  provide  as 
many  external  ECAMPs  as  possible,  and  all  available  AFMC 
external  ECAMPs  were  obtained  by  visiting  the  program 
manager. 

As  a  result  of  these  activities,  external  ECAMP 
findings  from  49  bases  were  collected.  Overall,  30  ACC,  12 
AFMC,  and  7  AMC  installations  are  represented  in  the  data 
set.  However,  only  35  of  these  evaluations,  22  ACC,  9  AFMC, 
and  4  AMC,  were  performed  at  least  cne  year  prior  to  the  NOV 
information  obtained  from  the  RCOs. 

NOV  Data.  Historical  NOV  data  was  acquired  from  the 
RCO  data  bases.  This  information  included  all  NOVs  received 
by  the  AF  prior  to  25  March  1993.  The  installation  specific 
NOV  data  was  then  used  to  assess  the  relationships  between 
NOVs  and  standardized  ECAMP  findings  for  1-year  periods 
prior  to  and  following  the  ECAMP  evaluations.  Descriptive 
statistics  were  also  used  to  evaluate  the  historical  NOV 
data  for  the  59  bases  in  the  potential  sample  population. 
This  information  was  used  to  determine  realistic  values  for 
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the  average  number  of  NOVs  used  in  the  ECAMP  effectiveness 
model . 

General  Methods 

This  section  outlines  the  general  methods  that  were 
used  to  achieve  the  three  research  objectives. 

Objective  1.  In  Chapter  II,  a  comprehensive  review  of 
the  literature  describing  the  development  of  the  Interim 
Environmental  Flight,  AFMC  Environmental  Management, 
Bioenvironmental  Engineering,  and  AFMC  Bioenvironmental 
Engineering  manpower  determinants  revealed  that  all  four 
developmental  procedures  systematically  apply  observational, 
data  collection,  and  statistical  methods  to  determine 
specific  manpower  requirements  based  on  the  varying  levels 
of  environmental  workload  at  dissimilar  installations.  In 
addition,  each  determinant  makes  extensive  efforts  to 
consider  all  relevant  workload  requirements  as  dictated  by 
installation  size,  mission,  and  local  regulatory 
enforcement. 

Thus,  the  literature  review  subjectively  demonstrated 
that  using  each  determinant  to  standardize  installation 
ECAMP  findings  for  the  respective  installations  where  they 
are  currently  being  applied  should  allow  unbiased 
comparisons  of  ECAMP  findings  from  dissimilar  installations 
using  the  same  manpower  determinants. 

Next,  the  manpower  authorizations  derived  using  the  two 
BEE  determinants  were  compared  to  the  manpower 
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authorizations  obtained  using  the  corresponding  Interim 
Environmental  Flight,  and  AFMC  Environmental  Management 
determinants  and  statistical  regression  analyses  were 
performed  to  determine  correlations  between  the 
standardizing  factors.  The  coefficients  of  determination 
and  the  information  obtained  during  the  literature  review 
were  then  be  used  to  select  the  standardizing  factors  for 
the  installations  in  the  sample  population.  These  factors 
were  then  used  to  standardize  historical  ECAMP  findings  from 
specific  AF  installations. 

Objective  2.  To  achieve  objective  2,  standardized 
ECAMP  findings  were  compared  to  historical  NOV  data  for  the 
respective  installations  in  a  pilot  study.  The  objective  of 
the  pilot  study  was  to  test  the  theory  that  there  should  be 
a  direct  relationship  between  the  number  of  standardized 
ECAMP  findings  and  environmental  compliance.  Regression 
analyses  were  used  to  assess  the  significance  of  the 
relationships . 

Based  on  these  results,  the  study  was  expanded  to  see 
if  significant  relationships  existed  between  both 
standardized  and  unstandardized  ECAMP  findings  and  NOVs  for 
35  AFMC,  ACC,  and  AMC  installations.  Again,  regression 
analyses  were  used  to  assess  the  significance  of  the 
relationships.  The  methodology  was  then  revised  and  an 
ECAMP  effectiveness  model  was  proposed. 

Objective  3.  The  proposed  ECAMP  effectiveness  model 
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was  then  tested  and  evaluated  using  24  bases  in  the  data  set 
with  external  ECAMP  evaluations  In  1991  or  1992  and  with  NOV 
data  from  both  1-year  prior  to  and  1-year  after  the 
evaluation.  First,  lines  representing  the  average  numbers 
of  NOVs  and  standardized  ECAMP  findings  were  drawn  on  the 
scatter  plots  to  define  four  ECAMP  effectiveness  quadrants. 
Next,  ECAMP  effectiveness  was  determined  by  plotting  NOVs 
from  1-year  prior  to  and  1-year  after  the  evaluations 
against  standardized  major  and  total  ECAMP  findings.  The 
results  were  then  evaluated  using  a  tabular  format  and  the 
standard  AF  rating  system  of  unsatisfactory,  marginal, 
satisfactory,  and  excellent.  Additional  analyses  using 
unstandardized  ECAMP  findings  demonstrated  the  flexibility 
of  the  proposed  model  and  suggested  that  the  BEE 
standardization  factor  does  account  for  the  dissimilarities 
between  large  and  small  installations  as  expected  and 
desired. 

Based  on  this  analysis,  the  proposed  ECAMP 
effectiveness  model  should  allow  AF  decision-makers  to 
quickly  and  easily  identify  installations  with  effective 
ECAMP  programs  and  those  bases  where  ECAMP  programs  need 
more  attention  and/or  resources.  Ultimately,  the  proposed 
ECAMP  effectiveness  model  should  assist  decision-makers  in 
identifying  the  installations  that  require  resource 
allocation  to  prevent  future  environmental  problems  and 
improve  environmental  compliance. 
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Sumroary 

The  research  methodology  described  in  this  chapter  uses 
readily  available  manpower  authorization,  ECAMP  finding,  and 
NOV  data  to  develop,  test,  evaluate,  and  revise  a 
methodology  to  assist  decision-makers  in  identifying  the 
installations  that  require  resource  allocation  to  prevent 
future  environmental  problems  and  improve  environmental 
compliance.  If  approved  and  implemented  by  the  AF,  the 
resulting  ECAMP  Effectiveness  Model  could  assist  decision¬ 
makers  in  properly  allocating  and  apportioning  scarce 
resources  among  installations  with  competing  environmental 
compliance  requirements.  Moreover,  using  the  proposed  model 
should  assist  decision-makers  in  improving  ECAMP 
effectiveness  thereby  reducing  NOVs  and  preventing  fines  and 
penalties  pursuant  to  the  Federal  Facilities  Compliance  Act. 
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IV.  Results  and  Analyses 


Overview 

This  chapter  contains  four  main  sections  labeled 
manpower  data  analysis,  pilot  study,  expanded  study,  and 
proposed  ECAMP  Effectiveness  Model.  Each  main  heading 
includes  three  sub-sections  that  provide  background 
information,  describe  and  discuss  research  results,  and 
summarize  section  findings.  Specific  section  previews  are 
also  provided  to  introduce  the  discussions  for  each  main 
area. 

Manpower  Data  Analysis 

This  section  provides  background  information  on 
regression  analysis,  describes  and  discusses  the  results  of 
six  manpower  regression  analyses,  summarizes  overall 
regression  analyses  findings,  and  concludes  that  BEE 
manpower  authorizations  will  be  used  as  the  primary 
standardization  factor  throughout  the  remainder  of  the 
research. 

Background.  The  first  step  in  the  data  analysis 
process  wes  to  evaluate  the  relationships  between  the 
respective  environmental  and  BEE  manpower  authorizations. 
Devore  states  that. 
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Regression  analysis  is  the  part  of  statistics  that 
deals  with  investigation  of  the  relationship  between 
two  or  more  variables  related  in  a  nondeterministic 
fashion.  (20:454) 

One  of  the  main  purposes  of  regression  analysis  is  to 
estimate  or  predict  values  for  the  dependent  variable  (Y) 
given  values  for  the  independent  variable  (X)  (41:259). 
Using  the  simple  linear  regression  model,  a  deterministic 
mathematical  relationship  can  then  be  established  between 
the  two  variables  in  the  form  of  a  straight  line  described 
by  the  equation  E(Y)  =A(X)  +B,  where  E(Y)  is  the  expected 
value  of  the  dependent  variable  Y,  A  is  the  slope,  and  B  is 
the  y-intercept  (20:454) .  Coefficients  of  determination, 
denoted  by  r^,  are  then  calculated. 

Results.  The  resulting  r^  values  describe  the 
proportion  of  observed  variation  in  Y  that  can  be  explained 
by  variations  in  X.  For  example,  an  r^  value  of  0.9  means 
that  90  percent  of  the  variations  in  Y  can  be  explained  by 
variations  in  X  (20:467).  The  data  used  in  the  manpower 
regression  analyses  is  provided  in  Appendix  A,  and  Figure  3 
graphically  illustrates  the  regression  analysis  of  the 
manpower  authorization  data  for  27  ACC,  12  AFMC,  and  12  AMC 
installations . 
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The  value  of  0.8261  suggests  that  approximately  82%  of 
the  variations  in  environmental  manpower  authorizations  (Y) 
can  be  explained  by  the  variations  in  BEE  manpower 
authorizations  (X) .  This  is  a  fairly  high  correlation  given 
that  the  data  includes  manpower  authorizations  from  three 
commands,  derived  using  four  manpower  determinants  as 
discussed  in  Chapter  II.  In  addition,  the  majority  of  the 
environmental  manpower  values  (27  ACC  and  12  AMC)  are  based 
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Figure  4.  Manpower  Regression  Analysis,  12  AFMC 
Installations 

Six  of  the  twelve  data  points  in  Figure  4  are  based  on  the 
EM  and  BEE  manpower  determinants  developed  specifically  for 
the  former  logistics  centers  as  discussed  in  Chapter  II. 
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The  remaining  six  data  points  are  based  on  the  actual  April 
1993  EM  and  BEE  manpower  authorizations  as  obtained  from  the 
MAJCOM  offices.  As  previously  alluded  to,  the  r^  value  for 
the  12  AFMC  bases  decreased  to  0.6741  as  compared  to  0.8261 
for  all  51  installations.  The  lower  r^  values  are  not 
unexpected  due  to  the  relative  young  age  and  developing 
status  of  AFMC  environmental  organizations.  To  gain  a 
better  understanding  of  the  dynamic,  evolving  nature  of  both 
AF  and  AFMC  environmental  organizations,  two  additional 
regression  analyses  were  performed  using  AFMC  manpower  data 
from  specific  time  periods.  The  actual  data  used  in  these 
analyses  are  presented  in  Appendix  B. 

First,  the  historical  AFMC  environmental  manpower 
authorizations  from  April  1992  are  plotted  and  analyzed. 
During  the  one  year  period  between  April  1992  and  April 
1993,  nine  AFMC  environmental  organizations  experienced 
increases  in  manpower  authorizations,  two  had  authorizations 
remain  the  same,  and  one  organization  experienced  a  slight 
manpower  authorization  decrease  while  BEE  manpower  remained 
constant.  The  r^  value  for  this  analysis  was  0.5550  as 
compared  to  the  0.6741  value  calculated  using  the  April  1993 
data.  This  improvement  seems  to  suggest  that  the 
relationship  between  AFMC  environmental  manpower  and  AFMC 
BEE  manpower  is  becoming  stronger  as  the  environmental 
organizations  mature.  Figure  5  displays  the  results  of  the 
first  analysis  using  the  April  1992  AFMC  manpower  data. 
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Figure  5.  Regression  Analysis  Using  April  1992  Data 


Next,  the  actual  number  of  AFMC  personnel  assigned  as 
of  April  1993  was  used  in  the  regression  analysis.  An 
examination  of  this  data  reveals  that  eight  installations 
had  fewer  people  assigned  then  authorized  while  two  had  more 
and  two  were  manned  at  the  required  level.  Figure  6  shows 
the  plot  and  regression  analysis  results  using  the  actual 
AFMC  manpower  data  from  April  1993. 
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Figure  6.  Regression  Analysis,  12  AFMC  Bases  Actual  Manning 
April  1993 


The  r^  value  for  the  actual  environmental  manning  analysis 
was  0.6725  as  compared  to  the  0.6741  value  calculated  using 
the  required  manning  over  the  same  time  period.  Although 
only  two  of  twelve  installations  were  actually  manned  at 
authorized  levels,  the  fact  that  these  two  r^  values  are 
essentially  the  same  could  suggest  that  true  workload 
conditions  at  the  installations  are  causing  environmental 
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manning  to  change  in  a  fashion  similar  to  the  maturation 
process  previously  experienced  by  BEE  manning. 

The  hypothesis  that  the  relationship  between 
environmental  manpower  and  BEE  manpower  will  become  stronger 
as  the  environmental  organization  matures  could  also  be 
supported  by  the  ACC  and  AMC  command  specific  manpower 
regression  analyses.  Plots  and  the  regression  analysis 
results  for  the  27  ACC  and  12  AMC  installations  are  shown  in 
Figures  7  and  8  respectively. 
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Figure  7.  Manpower  Regression  Analysis,  27  ACC 
Installations 
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y  -  0^126(X)  4^  6^129 
R^QUARED:  0.3200,  N  -  12 
STD  ERRORS:  A  -  0.23677,  B  -  2.81703 

Figure  8.  Manpower  Regression  Ana'ysis,  12  AMC 
Installations 

Although  the  ACC  and  AMC  r^  values  of  0.1611  and  0.32  09 
indicate  that  only  16  to  32%  of  the  variations  in 
environmental  manpower  authorizations  (Y'  can  be  explained 
by  the  variations  in  BEE  manpower  authorizations  (X) ,  low  r^ 
values  were  not  unexpected  since  all  39  data  points  are 
based  on  the  interim  environmental  flight  manpower 
determinant  which  is  still  developing  and  changing.  In 
effect,  these  low  r^  values  could  support  the  theory  that 
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the  relationship  between  environmental  and  BEE  manpower  is 
becoming  stronger  as  the  environmental  organizations  mature 
if  the  correlation  increases  with  time  as  appears  to  have 
been  the  case  with  the  AFMC  environmental  management 
manpower  determinant. 

Summary .  The  regression  analysis  for  27  ACC,  12  AFMC, 
and  12  AMC  installations  resulted  in  a  coefficient  of 
determination  of  0.8243.  This  value  suggests  that  the 
overall  or  macro-level  relationship  between  BEE  manpower  and 
environmental  manpower  is  relatively  strong.  Although 
command  specific  or  micro-level  regression  analyses  resulted 
in  significantly  lower  coefficients  of  determination,  these 
values  were  not  unexpected  based  on  the  relative  young  age 
and  developing  status  of  tne  various  environmental 
organizations  and  the  respective  manpower  determinants.  The 
r^  values  for  the  ACC  and  AMC  analyses  where  all  the  data 
points  are  based  on  the  new  and  changing  interim 
environmental  flight  manpower  determinant  were  the  lowest  at 
0.16  and  0.32  respectively.  However,  the  r^  value  for  the 
more  established  AFMC  environmental  data  increased  to 
0.6741.  Furthermore,  a  more  detailed  analysis  of  the 
historical  AFMC  environmental  authorizations  suggests  that 
the  relationship  between  AFMC  environmental  manpower  and 
AFMC  BEE  manpower  is  becoming  stronger  as  the  environmental 
organizations  mature. 

Overall,  the  regression  analyses  of  the  manpower 
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authorization  data  indicate  that  the  relationship  between 
BEE  manpower  and  environmental  manpower  is  relatively  strong 
at  the  macro-level  and  weak  but  possibly  becoming  stronger 
at  the  micro-level.  In  general,  these  regression  analyses 
support  the  Chapter  II  conclusion  that  the  BEE  manpower 
determinant  may  be  the  best  standardizing  factor  since  it 
has  matured  over  time  with  the  developing  environmental 
legislation.  Thus,  BEE  manpower  authorizations  will  be  used 
as  the  primary  standardization  factor  in  the  pilot  study  and 
throughout  the  remainder  of  this  research. 

Pilot  Study 

This  section  provides  background  Information  on  the 
pilot  study  objective,  describes  and  dir  ^.usses  the  results 
of  six  regression  analyses  performed  to  assess  the 
relationship  between  NOVs  and  standardized  ECAMP  findings, 
summarizes  overall  regression  analyses  findings,  and 
concludes  that  an  expanded  study  is  justified. 

Background.  The  objective  of  the  pilot  study  was  to 
test  the  theory  that  there  should  be  a  direct  relationship 
between  the  number  of  standardized  ECAMP  findings  and 
environmental  compliance.  It  was  proposed  that  the  number 
of  NOVs  received  1-year  after  an  external  ECAMP  evaluation 
should  be  dependent  upon  the  number  of  findings  contained  in 
the  ECAMP  report  after  the  findings  had  been  standardized 
using  BEE  manpower  authorizations.  To  test  this  theory,  all 
available  reports  for  AFMC  external  ECAMP  evaluations 
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performed  1-year  prior  to  the  date  of  the  NOV  data  provided 
by  the  RCOs,  25  March  1993,  were  obtained  by  visiting  the 
AFMC  ECAMP  program  manager.  Based  on  this  criteria,  the 
numbers  of  major  and  total  ECAMP  findings  were  then 
extracted  from  the  reports  for  nine  AFMC  installations. 

These  numbers  were  then  divided  by  the  number  of  BEE 
manpower  authorizations  at  the  respective  bases  to  produce 
standardized  findings.  The  number  of  NOVs  received  by  the 
respective  installations  within  1-year  following  the  ECAMP 
evaluation  were  then  obtained  from  the  RCO  data  and  plotted 
against  the  standardized  findings.  Regression  analyses  were 
then  performed  to  investigate  the  relationship  between  the 
number  of  NOVs  (Y)  and  the  number  of  standardized  ECAMP 
findings  (X) . 

Results.  The  results  of  the  six  regression  analyses 
performed  to  assess  this  relationship  are  described  and 
discussed  in  this  sub-section,  and  the  AFMC  data  used  in  the 
pilot  study  regression  analyses  are  shown  in  Table  9. 
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TABLE  9 


AFMC  PILOT  STUDY  DATA 


(1) 

BASE 

(2) 

MAJOR 

ECAMP 

FINDINGS 

<3) 

TOTAL, 

ECAMP 

FINDINGS 

(4) 

NOVs 

SINCE 

ECAMP 

(5) 

BEE 

STD 

FACTOR 

(6) 

BEE 

STD 

MAJOR 

(7) 

BEE 

STD 

TOTAL 

AFMC-1 

47 

82 

3 

9 

5.22 

9.11 

AFMC-3 

24 

56 

1 

44 

0.55 

1.27 

AFMC- 4 

68 

101 

5 

51 

1.33 

1.98 

AFMC- 5 

54 

74 

2 

16 

3.38 

4.63 

AFMC- 6 

55 

113 

0 

43 

1.28 

2.63 

AFMC-7 

67 

85 

7 

32 

2.09 

2.66 

AFMC-9 

44 

88 

5 

47 

0.94 

1.87 

AFMC- 10 

119 

156 

3 

27 

4.41 

5.78 

AFMC-12 

48 

75 

4 

51 

0.94 

1.47 

The  values  in  columns  six  and  seven  of  Table  9  are  obtained 
by  dividing  the  second  and  third  columns  by  the  BEE 
standardization  factors  in  column  five.  The  number  of  NOVs 
in  column  four  was  then  plotted  against  the  number  of 
standardized  findings  in  the  last  two  columns  and  regression 
analyses  were  performed.  Plots  and  regression  analysis 
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results  for  the  nine  AFMC  installations  in  the  pilot  study 
are  shown  in  Figures  9  and  10.  Specifically,  Figure  9  plots 
NOVs  against  standardized  major  ECAMP  findings  while  Figure 
10  displays  the  relationship  between  NOVs  and  standardized 
total  ECAMP  finding. 
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Figure  9.  Regression  Analysis,  Major  ECAMP  Findings 
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Y  -  •  0.1068()q  +  3.7063 
R-8QUARE0:  0.0161,  N  -  9 
STD  ERRORS:  A  -  0.31606,  B  -  1.34614 

Figure  10.  Regression  Analysis,  Total  ECAMP  Findings 

The  very  low  r^  values  of  0.0016  and  0.0161  seem  to 
indicate  that  the  number  of  NOVs  is  not  dependent  on  the 
number  of  standardized  ECAMP  findings.  However,  five  of  the 
installations  in  the  pilot  study  were  former  AF  Logistics 
Command  (AFLC)  bases,  two  were  former  AF  Systems  Command 
(AFSC)  bases,  and  one  was  previously  a  Military  Airlift 
Command  (MAC)  installation.  Since,  as  discussed  in 
Chapter  II,  the  BEE  standardization  factor  is  based  on  the 
AFMC  specific  BEE  manpower  determinant  for  the  former  AFLC 
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bases  and  the  standard  BEE  manpower  determinant  for  the 
other  three  facilities,  additional  analyses  were  performed 
using  only  the  data  for  the  former  AFLC  installations.  The 
results  of  these  analyses  are  shown  in  Figures  11  and  12. 


Y  -  a.osirpo  +  0.07U 
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Figure  11.  Regression  Analysis,  Major  ECAMP  Findings  Former 
AFLCs  Only 
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Figure  12.  Regression  Analysis,  Total  ECAMP  Findings  Former 
AFLCs  Only 


Once  again,  the  very  low  r^  values  of  0.3546  and  0.0363 
seem  to  indicate  that  the  number  of  NOVs  is  not  dependent  on 
the  number  of  standardized  ECAMP  findings.  However,  closer 
inspection  of  the  two  plots  and  the  data  in  Table  9  reveals 
that  although  base  AFMC-6  has  1.28  and  2.63  standardized 
ECAMP  major  and  total  findings  respectively,  it  has  no  NOVs. 
This  absence  of  NOVs  appears  unusual  since  other  former 
AFLCs  with  similar  numbers  of  standardized  findings  have 
from  five  to  seven  NOVs.  In  addition,  a  review  of  the  NOV 
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databases  indicates  that  the  other  five  former  AFLCs  have  an 
average  of  5.6  NOVs  per  installation  from  January  1991 
through  December  1992  while  base  AFMC-6  has  no  NOVs  over  the 
same  time  period.  Furthermore,  the  other  five  installations 
have  an  average  of  8.2  NOVs  in  the  database  while  base 
AFMC-6  has  only  three.  From  a  statistical  perspective,  the 
95  percent  confidence  interval  for  NOVs  at  the  six  former 
AFLCs  is  0.8769  to  6.4564.  Thus,  the  absence  of  NOVs  at 
base  AFMC-6  appears  to  be  anomalous  and  additional 
regression  analyses  were  performed  on  the  remaining  former 
AFLC  installations  without  the  base  AFMC-6  data.  The 
results  of  these  analyses  are  shown  in  Figures  13  and  14. 
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NOVS 
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Figure  13.  Regression  Analysis,  Major  ECAMP  Findings 
Former  AFLCs  Without  Base  AFMC-6 
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Figure  14.  Regression  Analysis,  Total  ECAMP  Findings 
Former  AFLCs  Without  Base  AFMC-6 


The  vastly  improved  r^  values  of  0.7916  and  0.8421 
suggest  that  approximately  80%  of  the  variations  in  NOVs  at 
these  five  installations  can  be  explained  by  the  variations 
in  ECAMP  findings  standardized  using  BEE  manpower 
authorizations.  Similar  analyses  using  unstandardized 
findings  resulted  in  r^  values  of  0.7748  and  0.6011  for 
major  and  total  ECAMP  findings  respectively.  Thus,  using 
standardized  findings  seems  to  result  in  somewhat  better 
coefficients  of  determination.  In  general,  the  positive 
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slope  of  both  sets  of  regression  lines  indicates  that  as  the 
number  of  standardized  ECAMP  findings  increases,  so  does  the 
number  of  NOVs.  This  positive  correlation  also  suggests 
that  although  the  ECAMP  evaluations  are  accurately 
identifying  compliance  problems,  the  ECAMP  findings  are  not 
being  effectively  used  to  identify  the  management  actions 
necessary  to  prevent  NOVs.  Since  the  former  AFLCs 
installations  account  for  a  substantial  portion  of  the 
overall  AFMC  environmental  budget,  this  correlation  could  be 
significant. 

Summary .  In  contrast  to  the  manpower  authorization 
regression  analyses  which  displayed  a  relatively  strong 
relationship  between  BEE  manpower  and  environmental  manpower 
at  the  macro-level  and  weak  relationship  at  the  micro-level, 
the  pilot  study  NOV  regression  analyses  indicated  a  very 
weak  relationship  at  the  macro-level,  for  nine  bases 
previously  associated  with  three  commands,  and  a  strong 
relationship  at  the  micro-level  for  five  similar  bases  from 
the  same  command.  Although  the  majority  of  the  pilot  study 
coefficients  of  determination  were  very  low,  the  0.79  and 
0.84  r^  values  for  the  five  former  AFLC  installations  are 
potentially  significant  and  clearly  warrant  followup.  In 
addition,  the  possibility  of  similar  relationships  on  a 
larger  scale  justifies  expanding  this  study  to  installations 
in  other  commands. 
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Expanded  Study 

This  section  provides  background  information  on  the 
purpose  of  the  expanded  study,  describes  and  discusses  the 
results  of  twenty  regression  analyses  performed  to  assess 
the  relationships  between  NOVs  and  both  standardized  and 
unstandardized  ECAMP  findings  on  the  macro-  and  micro¬ 
levels,  introduces  the  proposed  ECAMP  Effectiveness  Model, 
and  summarizes  the  expanded  study  findings. 

Background.  The  purpose  of  the  expanded  study  was  to 
investigate  the  relationships  between  NOVs  and  both 
standardized  and  unstandardized  ECAMP  findings  on  the  macro- 
and  micro-  levels  with  installations  from  more  than  one 
major  command.  The  specific  objectives  of  this  portion  of 
the  research  were  to  determine  if  significant  relationships 
exist  between  standardized  and  unstandardized  ECAMP  findings 
and  NOVs  on  the  macro-level  for  35  installations  from  AFMC, 
ACC,  and  AMC  and  at  the  micro-level  for  22  ACC  and  4  AMC 
facilities.  To  assess  these  relationships,  numerous 
regression  analyses  were  performed  using  the  ECAMP  finding 
and  NOV  data  in  Appendix  C.  Overall,  22  ACC,  9  AFMC,  and  4 
AMC  installations  are  represented  in  the  data  set.  The 
standardization  procedures  used  were  the  same  as  those 
outlined  in  the  pilot  study,  and  all  regression  analysis 
results  describe  the  relationship  between  the  number  of  NOVs 
(Y) ,  and  the  number  of  standardized  ECAMP  findings  (X) . 

Results .  The  regression  analysis  descriptions,  numbers 
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of  datum  points  used,  resulting  values,  and  regression 
line  equations  for  standardized  ECAMP  findings  are  presented 
in  Table  10. 
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TABLE  10 

EXPANDED  STUDY  REGRESSION  ANALYSES  RESULTS  STANDARDIZED 
ECAMP  FINDINGS 


REGRESSION 

NUMBER 

R-’ 

REGRESSION 

ANALYSIS 

DATA 

LINE 

DESCRIPTION 

PTS 

VALUES 

EQUATIONS 

NOVs  VS  Std  Major  ECAMP 
Findings  All  Expanded 
Stooy  Bases 


NOVs  VS  std  Total  ECAMP 
Findings  All  Expanded 
Study  Bases 


NOVs  VS  Std  Major  ECAMP 
Findings  ACC  and  AMC 
Bases  Only 


NOVs  VS  Std  Total  ECAMP 
Findings  ACC  and  AMC 
Bases  Only 


NOVs  VS  Std  Major  FOAMP 
Findings  ACC  Bases 
Only 


NOVs  VS  Std  Total  ECAMP 
Findings  ACC  Bases 
Only 


NOVs  VS  Std  Major  ECAMP 
Findings  1992  ECAMPs 
Only ,  ALL  Comiiands 


0.0264  Y  =  -  0.195(X)  +  2.473 


=  -  0.2355(X)  +  3.243 


0.0096  y  =  -  0.1044(X)  +  1.745 


0.0406  y  =  -  0.1527(X)  +  2.399 


0.0029  y  =  -  O-OECT^yj  +  1.608 


0.0564  y  =  -  0.1616(X)  +  2.534 


0.3356  y  =  -  0.6690(X)  +  5.096 


10 
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TABLE  10 


EXPANDED  STUDY  REGRESSION  ANALYSES  RESULTS  STANDARDIZED 
ECAMP  FINDINGS 


REGRESSION 

ANALYSIS 

DESCRIPTION 

NUMBER 

DATA 

PTS 

R- 

VALUES 

REGRESSION 

LINE 

EQUATIONS 

NOVs  VS  Std  Total  ECAMP 

Findings  1992  ECAMPs 

Only,  All  Commands 

10 

0.4708 

Y  =  -  0.4379(X)  +  5.47 

NOVs  VS  Std  Major  ECAMP 

Findings  1991  ECAMPs 

Only,  All  Commands 

14 

0.0840 

y  =  -  0.2887(X)  +  2.604 

NOVs  VS  Std  Total  ECAMP 

Findings  1991  ECAMPs 

Only,  All  Commands 

14 

0.1680 

y  =  -  0.2156(X)  +  2.958 

As  the  values  in  Table  10  indicate,  no  significant 
relationships  between  NOVs  and  standardized  ECAMP  findings 
were  identified  at  either  the  macro-  or  micro-  levels  in  the 
expanded  study.  However,  significant  increases  in  r^  values 
for  the  1992  ECAMP  evaluations  over  the  r^  values  for  the 
1991  evaluations  were  noted,  and  in  contrast  to  the  pilot 
study,  all  the  regression  lines  in  Table  10  have  negative 
slopes. 

Since  no  significant  relationships  were  identified 
using  standardized  ECAMP  findings,  similar  regression 
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analyses  were  performed  using  the  unstandardized  findings  to 
determine  if  there  was  a  significant  difference  in  the 
relationships  between  standardized  ECAMP  findings  and  NOVs 
and  unstandardized  ECAMP  findings  and  NOVs.  These  analyses 
were  performed  using  the  same  ECAMP  finding  and  NOV  data 
that  were  used  to  produce  the  results  in  Table  10;  however, 
the  findings  were  not  standardized.  The  results  of  these 
analyses  are  shown  in  Table  11. 
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TABLE  11 


EXPANDED  STUDY  REGRESSION  ANALYSES  RESULTS 
UNSTANDARDIZED  ECAMP  FINDINGS 


REGRESSION 

ANALYSIS 

DESCRIPTION 

NUMBER 

DATA 

PTS 

RE¬ 

VALUES 

REGRESSION 

LINE 

EQUATIONS 

NOVs  VS  Unstd  Major 

ECAMP  Findings  All 

Expanded  Study  Bases 

35 

0.1125 

Y  =  0.03(X)  +  0.8760 

NOVs  VS  Unstd  Total 

ECAMP  Findings  All 

Expanded  Study  Bases 

35 

0.0356 

Y  =  0.0129(X)  +  1.089 

NOVs  VS  Unstd  Major 

ECAMP  Findings  ACC  and 

AMC  Bases  Only 

26 

0.0023 

Y  =  5.8E-03(X)  +  1.270 

NOVs  VS  Unstd  Total 

ECAMP  Findings  ACC  and 

AMC  Bases  Only 

26 

0.0066 

Y  =  -  6.3E-03(X)  +  1.75 

NOVs  VS  Unstd  Major 

ECAMP  Findings  ACC 

Bases  Only 

22 

0.0056 

Y  =  9.1E-03(X)  +  1.106 

NOVs  VS  Unstd  Total 

ECAMP  Findings  ACC 

Bases  Only 

22 

0.0048 

Y  =  -  5.4E-03(X)  +  1.68 

TABLE  11 


EXPANDED  STUDY  REGRESSION  ANALYSES  RESULTS 
UNSTANDARDIZED  ECAMP  FINDINGS 


REGRESSION 

ANALYSIS 

DESCRIPTION 

NUMBER 

DATA 

PTS 

R- 

VALUES 

REGRESSION 

LINE 

EQUATIONS 

NOVs  VS  Unstd  Major 

ECAMP  Findings  1992 

ECAMPs  Only,  All 

Commands 

10 

0.0700 

Y  =  0.022(X)  + 

1.5257 

NOVs  VS  Unstd  Total 

ECAMP  Findings  1992 

ECAMPs  Only,  All 

Commands 

10 

0.0157 

Y  =  8.86E-03(X) 

+  1.92 

NOVs  VS  Unstd  Major 

ECAMP  Findings  1991 

ECAMPs  Only,  All 

Commands 

14 

0.0893 

Y  =  0.0337(X)  + 

0.5277 

NOVs  VS  Unstd  Total 

ECAMP  Findings  1991 

ECAMPs  Only,  All 

Commands 

14 

0.0030 

Y  =  4.3E-03(X)  + 

1.4404 

Once  again,  no  significant  relationships  between  NOVs 
and  unstandardized  ECAMP  findings  were  identified  at  either 
the  macro-  or  micro-  levels.  Overall,  seven  of  ten  r^ 
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values  were  higher  with  standardized  findings  than  with 
unstandardized  findings.  In  addition,  the  significant 
increases  in  r^  values  for  the  1992  data  that  were  present 
using  standardized  findings  were  absent  when  unstandardized 
findings  were  used.  The  most  notable  difference  however, 
was  the  change  in  slope  of  the  regression  lines. 

All  ten  regression  lines  were  negatively  sloped  when 
standardized  ECAMP  findings  were  used  while  seven  of  ten 
regression  lines  were  positively  sloped  with  unstandardized 
ECAMP  findings.  At  first,  this  difference  in  slopes  appears 
significant  since  the  slope  of  the  regression  line  should 
reflect  overall  ECAMP  effectiveness.  Negatives  slopes  would 
suggest  that  as  the  number  of  ECAMP  findings  increases,  the 
number  of  NOVs  decreases,  and  positive  slopes  indicate  that 
as  the  number  of  ECAMP  findings  increases,  the  number  of 
NOVs  also  increases.  An  effective  ECAMP  should  be 
characterized  by  a  negative  slope  while  a  positive  slope 
would  be  indicative  of  an  ineffective  ECAMP.  However,  the 
slopes  of  all  twenty  regression  lines  are  between  -0.669  and 
0.0337  so  the  difference  in  slope  is  not  as  significant  as 
it  first  appeared.  In  addition,  the  low  r^  values  for  all 
twenty  analyses  make  it  unrealistic  to  draw  conclusions 
about  ECAMP  effectiveness  from  these  results. 

Although  no  metrics  for  measuring  ECAMP  effectiveness 
that  use  either  standardized  or  unstandardized  ECAMP 
findings  and  historical  NOVs  are  currently  available,  closer 
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inspection  of  the  scatter  plot  for  all  the  data  in  the 
expanded  study,  Figure  15,  resulted  in  a  theoretical 
framework  for  the  development  of  such  a  ECAMP  Effectiveness 
Model  or  metric. 


Figure  15,  Scatter  Plot,  All  Expanded  Study  Data 


In  general,  the  proposed  ECAMP  Effectiveness  Model  or  metric 
involves  dividing  the  scatter  plot  into  four  quadrants  by 
plotting  the  lines  for  the  average  number  of  NOVs  and 
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standardized  ECAMP  findings.  In  this  case,  the  four 
quadrants  are  constructed  by  plotting  the  lines  Y  =  2.0571 
and  X  =  2.8817  as  shown  in  figure  16. 
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Figure  16.  ECAMP  Effectiveness  Model  Quadrants 


This  four  quadrant  approach  to  ECAMP  effectiveness  is 
patterned  after  the  two-factor  theory  of  leadership 
developed  by  Fleishman  and  his  associates  at  Ohio  State 
University  (OSU)  in  the  1950s  (26:375).  In  the  OSU  studies, 
the  two  leadership  effectiveness  factors  of  initiating 
structure  (Y)  and  consideration  (X)  were  assessed  using 


89 


questionnaires  and  plotted  with  the  premise  that  a  high 
degree  of  consideration  and  a  high  degree  of  initiating 
structure  (High-High  or  Quadrant  2)  was  best  (26:375-376). 
Similarly,  the  proposed  model  assesses  and  plots  the  two 
ECAMP  effectiveness  factors  of  NOVs  (Y)  and  standardized 
ECAMP  findings  (X)  with  the  premise  that  fewer  NOVs  and 
fewer  standardized  ECAMP,  (Low-Low  or  Quadrant  4)  are  best. 
Although  the  expanded  study  did  not  yield  significant 
coefficients  of  determination,  the  proposed  ECAMP 
Effectiveness  Model  could  be  used  to  assist  AF  decision¬ 
makers  in  identifying  the  installations  that  require 
resource  allocation  to  prevent  future  environmental  problems 
and  improve  environmental  compliance. 

Summary .  In  contrast  to  the  pilot  study,  no 
significant  relationships  between  NOVs  and  standardized  or 
unstandardized  ECAMP  findings  were  identified  at  either  the 
macro-  or  micro-  levels  in  the  expanded  study.  However,  a 
detailed  review  of  the  expanded  study  data  did  result  in  a 
theoretical  framework  for  a  proposed  ECAMP  Effectiveness 
Model  or  metric.  The  proposed  model  is  patterned  after  the 
two-factor  theory  of  leadership,  and  the  model  is  based  on 
the  premise  that  fewer  NOVs  and  fewer  ECAMP  findings  are 
best.  This  premise  is  based  on  logic  which  is  described  in 
greater  detail  in  the  next  section  on  the  proposed  ECAMP 
Effectiveness  Model. 
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Proposed  ECAMP  Effectiveness  Model 

This  section  provides  background  information  on  ECAMP 
effectiveness  and  a  theoretical  ECAMP  maturation  process, 
describes  the  use  of  the  proposed  ECAMP  Effectiveness  Model, 
discusses  model  results,  summarizes  model  findings,  and 
concludes  that  implementing  the  proposed  model  could  assist 
AF  decision-makers  in  ensuring  that  scarce  resources  are 
properly  allocated  and  apportioned  among  installations  with 
competing  environmental  compliance  requirements. 

Background.  In  general,  an  effective  ECAMP  program 
should  be  characterized  by  a  negative  correlation  between 
NOVs  and  standardized  ECAMP  findings.  In  other  words,  as 
the  number  of  standardized  ECAMP  findings  increases,  more 
compliance  problems  are  identified,  corrected,  and  prevented 
so  the  number  of  NOVs  should  decrease.  Theoretically,  a 
perfectly  effective  and  properly  administered  environmental 
auditing  program  such  as  ECAMP  should  result  in 
progressively  fewer  ECAMP  findings  and  ultimately  a  gradual 
reduction  in  the  number  of  NOVs  as  all  potential  compliance 
problems  are  identifitJ,  corrected,  and  subsequently 
prevented . 

ECAMP  should  evolve  and  mature  over  time.  Since  ECAMP 
is  a  decentralized  program,  the  maturation  process  may  vary 
between  commands  and  from  one  installation  to  the  next.  In 
addition,  other  factors  such  as  manpower  shortages  and 
changes  in  environmental  regulatory  requirements  may  cause 
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ECAMP  effectiveness  to  fluctuate  between  effectiveness 
quadrants  with  fewer  ECAMP  findings  and  NOVs  indicating  an 
effective  ECAMP.  In  theory  however,  the  ECAMP  maturation 
process  would  be  expected  to  progress  through  four  distinct 
phases.  Installations  in  phase  I  of  the  ECAMP  evolutionary 
process  will  be  distinguished  by  more  NOVs  and  fewer 
standardized  ECAMP  findings  than  expected  since  the  program 
is  immature  and  base  personnel  are  inexperienced.  As  the 
ECAMP  matures  and  personnel  become  more  familiar  with 
identifying  compliance  problems,  the  program  should  progress 
to  phase  II  which  will  be  characterized  by  both  more  NOVs 
and  ECAMP  findings.  In  phase  III  of  the  ECAMP  maturation 
process,  installation  personnel  become  more  adept  at 
correcting  ECAMP  findings  before  NOVs  occur,  and 
installations  in  this  phase  will  be  identified  by  fewer  NOVs 
and  more  ECAMP  findings  then  expected.  Finally,  in  phase  IV 
of  the  hypothetical  ECAMP  maturation  process  the  program  is 
effectively  identifying,  correcting,  and  preventing 
compliance  problems.  As  a  result,  installations  in  phase  IV 
will  be  recognized  by  both  fewer  NOVs  and  fewer  ECAMP 
findings.  The  four  phases  described  above  also  correspond 
to  the  four  quadrants  as  depicted  in  Figure  17. 
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Figure  17.  ECAMP  Effectiveness  Model  Quadrants  and  Phases 

Since  the  phase  or  quadrant  boundaries  are  determined 
by  the  user,  simply  plotting  NOVs  against  standardized  ECAMP 
findings  and  drawing  lines  representing  AF  or  command 
averages  will  provide  the  AF  decision-maker  with  a  quick 
visual  indication  of  an  installation's  relative  ECAMP 
effectiveness.  In  addition,  the  proposed  model  is  very 
flexible,  and  it  can  be  used  both  prospectively  and 
retrospectively  by  plotting  NOVs  prior  to  and  following  any 
ECAMP  evaluation.  The  use  of  the  proposed  ECAMP 
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Effectiveness  Model  is  described  in  the  results  section  that 
follows . 

Results .  In  order  to  obtain  relative  ECAMP 
effectiveness  results  from  the  proposed  model,  the  first 
step  was  to  determine  the  average  numbers  of  NOVs  and 
standardized  ECAMP  findings  which  would  be  plotted  on  the 
scatter  diagram  to  define  the  four  phases  or  ECAMP 
effectiveness  quadrants.  To  ensure  that  the  values  used  in 
the  model  were  realistic,  the  historical  NOV  data  for  the  12 
AFMC,  34  ACC,  and  13  AMC  installations  used  in  this  study 
were  compiled  as  shown  in  Appendix  D.  Similarly,  the  ECAMP 
finding  data  for  the  same  installations  were  organized  as 
shown  in  Appendix  E.  This  information  was  then  used  to 
calculate  the  average  numbers  of  NOVs  and  standardized  ECAMP 
findings.  These  results  and  other  pertinent  descriptive 
statistics  are  displayed  in  Tables  12  and  13. 
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TABLE  12 


NOV  DATA  DESCRIPTIVE  STATISTICS 


DESCRIPTIVE 

NOVs 

NOVs 

NOVs 

NOVs 

STATISTICS 

1989 

1990 

1991 

1992 

DATA  POINTS 

59 

59 

59 

59 

LOWER  95%  C.I. 

0.2023 

0.6789 

1.0363 

1.2347 

MEAN 

0.3898 

0.9830 

1.5423 

1.6610 

UPPER  95%  C.I. 

0.5773 

1.2871 

2 . 0483 

2 . 0873 

STD  DEVIATION 

0.7196 

1.1669 

1.9415 

1.6359 

MINIMUM 

0.0 

o 

• 

o 

0.0 

0.0 

MEDIAN 

o 

o 

1.0 

1.0 

1.0 

MAXIMUM 

3.0 

5.0 

8.0 

7.0 
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TABLE  13 

ECAMP  DATA  DESCRIPTIVE  STATISTICS 


STD 

STD 

STD 

STD 

MAJOR 

TOTAL 

MAJOR 

TOTAL 

DESCRIPTIVE 

FINDINGS 

FINDINGS 

FINDINGS 

FINDINGS 

STATISTICS 

ALL 

ALL 

1991-93 

1991-93 

DATA  POINTS 

58 

58 

47 

47 

LOWER  95%  C.I. 

3 . 6198 

6.8306 

4 . 0944 

7.3825 

MEAN 

4.3839 

8.1439 

4.9561 

8.9148 

UPPER  95%  C.I. 

5.1480 

9.4573 

5.8179 

10.447 

STD  DEVIATION 

2.9059 

4.9949 

2.9349 

5.2188 

MINIMUM 

0.69 

1.47 

0.69 

1.47 

MEDIAN 

4.0 

7.55 

4.92 

8.63 

MAXIMUM 

12.4 

20.13 

12.4 

20.13 

Based  on  these  results,  the  average  numbers  of  historical 
NOVs  and  standardized  ECAMP  findings  used  in  the  model  were 
1.6,  5,  and  9  for  NOVs,  standardized  major,  and  standardized 
total  findings  respectively. 

Next,  the  bases  in  the  data  set  with  ECAMP  evaluations 
in  1991  or  1992  and  with  NOV  data  for  both  1-year  prior  to 
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and  1-year  after  the  evaluation  were  identified.  The  NOV 
and  ECAMP  data  for  the  24  bases  meeting  these  requirements 
are  listed  in  Appendix  F.  This  information  was  then 
combined  with  the  average  numbers  of  NOVs  and  standardized 
ECAMP  findings  to  produce  the  ECAMP  Effectiveness  Model 
scatter  plots  in  Figures  18-21. 
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ST/VNDAHDIZEO  MAJOR  ECAMP  FINDINQ8 

Y  -  MEAN  NUMBER  OF  NOVs  -  1.6.  N  -  66 

X  -  MEAN  NUMBER  OF  STD  MAJOR  -  5,  N  -  47 

Figure  18.  ECAMP  Effectiveness  Model,  NOVs  1-Year  Prior 
Versus  Standardized  Major  Findings 

Figure  18  displays  relative  ECAMP  effectiveness  by 
plotting  NOVs  1-year  prior  to  the  ECAMP  evaluation  against 
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standardized  major  ECAMP  findings.  Of  the  24  bases  shown  on 
the  plot,  seven  fall  in  quadrant  1,  three  in  quadrant  2, 
three  in  quadrant  3,  and  eleven  in  quadrant  4.  Overall,  the 
10  bases  in  quadrants  1  and  2  received  more  NOVs  than  the 
a'  .rage  of  1.6  in  the  1-year  immediately  prior  to  the  ECAMP 
evaluation  while  the  14  bases  in  quadrants  3  and  4  received 
fewer  NOVs  than  average  over  the  same  time.  Similarly,  the 
17  bases  in  quadrants  1  and  4  had  fewer  major  ECAMP  findings 
than  average  while  the  7  bases  in  quadrants  2  and  3  received 
more  than  the  average  of  5. 

Figure  19  plots  the  NOVs  1-year  prior  to  the  same 
evaluation  against  the  standardized  total  ECAMP  findings. 
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Figure  19.  ECAMP  Effectiveness  Model,  NOVs  1-Year  Prior 
Versus  Standardized  Total  Findings 


This  plot  shows  eight  bases  in  quadrant  1,  two  in 
quadra.it  2,  four  in  quadrant  3,  and  ten  in  quadrant  4. 

These  findings  are  very  similar  to  those  plotted  in 
Figure  17,  and  the  number  of  installations  falling  into 
quadrants  1  and  2,  and  3  and  4  are  identical.  These 
similarities  seem  to  indicate  that  both  standardized  major 
and  standardized  total  findings  provide  similar  information 
about  ECAMP  effectiveness.  However,  plotting  both 
standardized  major  and  total  findings  could  provide 
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additional  insight  into  which  direction  an  installation's 
program  is  progressing  in  the  ECAMP  maturation  process. 

To  investigate  this  possibility,  the  same  standardized 
major  and  total  findings  were  plotted  against  NOVs  1-year 
after  the  ECAMP  evaluation.  The  number  of  NOVs  1-year  after 
the  ECAMP  evaluations  are  plotted  against  standardized  major 
ECAMP  findings  in  Figure  20. 
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Figure  20.  ECAMP  Effectiveness  Model,  NOVs  : -Year  After 
Versus  Standardized  Major  Findings 


The  scatter  plot  in  Figure  20  -hows  eleven  bases  in 
quadrant  1,  two  in  quadrant  2,  three  in  quadrant  3,  and 
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eight  in  quadrant  4.  This  is  a  significant  change  from 
Figure  18  where  seven,  three,  three,  and  eleven  bases  were 
in  quadrants  1-4  respectively.  To  see  if  total  findings 
reveal  similar  changes,  NOVs  1-year  after  the  ECAMP 
evaluations  are  plotted  against  standardized  total  findings 
in  Figure  21. 
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Figure  21.  ECAMP  Effectiveness  Model,  NOVs  1-Year  After 
Versus  Standardized  Total  Findings 


Figure  21  is  very  similar  to  Figure  20  with  eleven, 
two,  four,  and  eight  bases  in  quadrants  1-4  respectively. 
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Overall,  thirteen  bases  received  more  NOVs  1-year  after 
these  ECAMP  evaluations  than  average.  This  number  is  up 
from  the  ten  installations  which  received  more  NOVs  than 
average  1-year  prior  to  the  same  evaluations.  In  general, 
this  indicates  a  decrease  in  ECAMP  effectiveness  which 
parallels  the  increasing  number  of  NOVs  that  the  AF  is 
receiving . 

To  assist  AF  decision-makers  in  reducing  the  number  of 
NOVs,  the  proposed  ECAMP  Effectiveness  Model  must  provide 
information  identifying  both  installations  with  effective 
ECAMP  programs  and  those  where  programs  need  more  attention 
or  resources  to  prevent  future  environmental  problems  and 
improve  environmental  compliance.  To  demonstrate  how  the 
proposed  model  can  be  used  to  provide  decision-makers  with 
this  information,  the  data  in  Figures  18-21  are  presented 
and  analyzed  by  installation  in  Table  14. 


TABLE  14 


AFMC-1 


AFMC-3 


AFMC-4 


AFMC-5 


AFMC-6 


AFMC-? 


AFMC-9 


AFMC-10 


AFMC-12 


ACC- 3 


ACC- 4 


ACC- 5 


ACC- 6 


ACC- 8 


ACC- 10 


ACC-18 


EFFECT 

TREND 

OVERAIX 

RATINGS 

NO  CHANGE 

MARG/MARG 

NO  CHANGE 

EXCEL/EXCEL 

NO  CHANGE 

UNSAT/ UN SAT 

DECREASE 

EXCEL/UNSAT 

NO  CHANGE 

EXCEL/EXCEL 

DECREASE 

EXCEL/UNSAT 

NO  CHANGE 

UNSAT/UNSAT 

NO  CHANGE 

UNSAT/UNSAT 

DECREASE 

EXCEL/UNSAT 

NO  CHANGE 

UNSAT/UNSAT 

INCREASE 

UNSAT/EXCEL 

INCREASE 

MARG/ EXCEL 

DECREASE 

EXCEL/UNSAT 

INCREASE 

MARG /SAT 

NO  CHANGE 

EXCEL/EXCEL 

NO  CHANGE 

SAT /SAT 
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TABLE  14 

ECAMP  EFFECTIVENESS  DATA  ANALYSIS  FIGURE  18-21 


(1) 

BASE 

QUAD. 

l-YR 

PRIOR 

PIQ  18 

QUAD. 

l-YR 

PRIOR 

PIG  19 

QUAD. 

APTER 

l-YR 

PIG  20 

QUAD. 

APTER 

l-YR 

PIG  21 

EPPECT 

TREND 

OVERALL 

RATINGS 

ACC-24 

4 

4 

4 

4 

NO  CHANGE 

EXCEL/EXCEL 

ACC-28 

4 

4 

1 

1 

DECREASE 

EXCEL/UNSAT 

ACC-31 

3 

3 

3 

3 

NO  CHANGE 

SAT /SAT 

ACC-32 

4 

4 

4 

4 

NO  CHANGE 

EXCEL/EXCEL 

ACC-33 

3 

3 

O 

2 

DECREASE 

SAT/MARG 

AMC-1 

1 

1 

1 

1 

NO  CHANGE 

UNSAT/UNSAT 

AMC-9 

4 

4 

1 

1 

DECREASE 

EXCEL/UNSAT 

AMC-10 

1 

1 

4 

4 

INCREASE 

UNSAT/EXCEL 

As  anticipated,  Table  14  indicates  that  ECAMP 
effectiveness  like  environmental  compliance  is  dynamic  and 
changing.  Overall,  eleven  of  the  24  bases  analyzed  in  the 
table  displayed  changes  in  ECAMP  effectiveness  with  seven 
installations  exhibiting  effectiveness  decreases  and  four 
showing  ECAMP  effectiveness  increases  over  the  two  year 
period.  By  assigning  standard  AF  ratings  of  unsatisfactory, 
marginal,  satisfactory,  and  excellent  to  quadrants  1-4 


104 


respectively,  it  is  also  possible  to  determine  overall  ECAMP 
effectiveness  ratings  for  each  installation  during  the  two, 
1-year  periods.  By  using  this  rating  system,  five 
installations  were  identified  with  consistently 
unsatisfactory  or  ineffective  ECAMP  programs,  six  other 
programs  have  decreased  in  effectiveness  from  excellent  to 
unsatisfactory,  and  two  additional  programs  were  rated  as 
marginal.  In  effect,  analyzing  the  ECAMP  Effectiveness 
Model  data  in  Figures  18-21  using  the  tabular  format  and 
rating  system  described  above  clearly  identifies  the 
installations  with  effective  and  ineffective  ECAMP  programs. 
Moreover,  the  proposed  model  is  extremely  flexible,  and  it 
can  be  easily  adapted  to  reflect  changes  in  AF  or  MAJCOM 
averages  or  goals. 

To  demonstrate  the  flexibility  of  the  proposed  model 
and  provide  additional  insight  into  the  effects  and 
potential  benefits  of  standardizing  ECAMP  findings  using  BEE 
manpower  authorizations,  the  NOV  and  ECAMP  data  for  the  same 
24  bases  was  analyzed  again  using  unstandardized  ECAMP 
findings.  The  actual  data  used  in  the  analyses  is  provided 
in  Appendix  G,  and  Figure  22  displays  relative  ECAMP 
effectiveness  by  plotting  NOVs  l-year  prior  to  the  ECAMP 
evaluation  against  unstandardized  major  ECAMP  findings. 
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Figure  22.  ECAMP  Effectiveness  Model,  NOVs  l-Year  Prior 
Versus  Unstandardized  Major  Findings 


In  effect.  Figure  22  uses  exactly  the  same  data  as 
Figure  18  except,  the  ECAMP  findings  were  not  standardized 
using  BEE  manpower  authorizations,  and  the  mean  numbers  of 
unstandardized  major  and  total  ECAMP  findings  were  used  to 
define  the  quadrants.  Eight  of  the  twenty-four  bases 
plotted  in  Figure  22  fall  in  quadrant  1,  two  in  quadrant  2, 
five  in  quadrant  3,  and  nine  in  quadrant  4.  This  compares 
with  seven,  three,  three,  and  eleven  bases  in  quadrants  1-4 
respectively  when  standardized  findings  were  used.  Since 


106 


the  number  of  NOVs  plotted  for  each  base  remained  the  same, 
it  was  not  possible  for  a  base  to  move  from  quadrants  1  or  2 
to  quadrants  3  or  4  and  vice  versa.  However,  shifts  between 
quadrants  1  and  2  or  between  quadrants  3  and  4  were 
possible.  In  this  instance.  Figure  22  indicates  that  using 
unstandardized  findings  caused  one  base  to  move  from 
quadrant  2  to  quadrant  1  while  two  bases  shifted  from 
quadrant  4  to  quadrant  3.  In  all  three  cases,  theses  shifts 
indicate  a  relative  decrease  in  ECAMP  effectiveness. 

Similarly,  Figure  23  plots  the  NOVs  1-year  prior  to  the 
same  evaluation  against  the  unstandardized  total  ECAMP 
findings. 
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Figure  23.  ECAMP  Effectiveness  Model,  NOVs  1-Year  Prior 
Versus  Unstandardized  Total  Findings 

This  figure  shows  six,  four,  four,  and  ten  bases  in 
quadrants  1-4  respectively.  Comparing  these  results  to  the 
corresponding  standardized  findings  previously  depicted  in 
Figure  19  reveals  that  two  bases  moved  from  quadrant  1  to 
quadrant  2  when  unstandardized  findings  were  used.  In 
contrast  to  the  differences  just  described  between  Figures 
18  and  22,  the  shifts  between  Figures  19  and  23  indicate 
relative  ECAMP  effectiveness  increases. 

To  further  investigate  the  differences  in  overall  model 
effectiveness  ratings  between  standardized  and 
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unstandardized  findings,  the  same  unstandardized  major  and 
total  findings  were  plotted  against  NOVs  1-year  after  the 
ECAMP  evaluations.  Figures  24  and  25  plot  the  number  of 
NOVs  1-year  after  the  ECAMP  evaluation  against 
unstandardized  major  and  total  ECAMP  findings  respectively. 
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Figure  24.  ECAMP  Effectiveness  Model,  NOVs  1-Year  After 
Versus  Unstandardized  Major  Findings 
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Figure  25.  ECAMP  Effectiveness  Model,  NOVs  1-Year  After 
Versus  Unstandardized  Total  Findings 


Once  again,  several  bases  fall  in  different 
effectiveness  quadrants  when  unstandardized  findings  are 
used  as  compared  to  standardized  findings.  Specifically, 
Figure  20  which  uses  standardized  major  findings  shows  four 
bases  in  different  effectiveness  quadrants  than  Figure  24 
which  uses  unstandardized  major  findings.  Similarly,  four 
bases  also  fall  in  different  quadrants  when  using 
standardized  and  unstandardized  total  findings  as  depicted 
in  Figures  21  and  25.  To  better  illustrate  these 
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differences,  the  data  in  Figures  22-25  is  presented  and 
analyzed  by  installation  in  Table  15. 

TABLE  15 

ECAMP  EFFECTIVENESS  MODEL  DATA  ANALYSIS  FIGURES  22-25 


AFMC-1 


AFMC-3 


AFMC-4 


AFMC-5 


APMC-6 


AFMC-7 


AFMC-9 


AFMC-10 


AFMC-12 


ACC-3 


ACC- 4 


ACC- 5 


ACC-6 


ACC- 8 


QUAD. 

QUAD. 

QUAD. 

QUAD. 

EFFECT 

OVERALL 

l-YR 

1-YR 

AFTER 

AFTER 

TREND 

RATINGS 

PRIOR 

PRIOR 

1-YR 

1-YR 

FZO  22 

PIO  23 

FIO  24 

FIG  25 

NO  CHANGE 

UNSAT/UNSAT 

NO  CHANGE 

EXCEL/EXCEL 

NO  CHANGE 

MARG/MARG 

DECREASE 

SAT/MARG 

NO  CHANGE 

SAT /SAT 

DECREASE 

SAT/MARG 

NO  CHANGE 

MARG/MARG 

NO  CHANGE 

MARG/MARG 

DECREASE 

EXCEL /UNSAT 

NO  CHANGE 

UNSAT /UNSAT 

UNSAT /EXCEL 

INCREASE 

UNSAT/EXCEL 

DECREASE 

EXCEL/UNSAT 

INCREASE 

UNSAT /EXCEL 
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TABLE  15 

ECAMP  EFFECTIVENESS  DATA  ANALYSIS  FIGURE  22-25 


(1) 

BASS 

QUAD. 

1-YR 

PRIOR 

FIO  22 

QUAD. 

1-YR 

PRIOR 

PIG  23 

QUAD. 

AFTER 

1-YR 

FIO  24 

QUAD. 

AFTER 

1-YR 

FIG  25 

EFFECT 

TREND 

OVERALL 

RATINGS 

ACC- 10 

4 

4 

4 

4 

NO  CHANGE 

EXCEL/EXCEL 

ACC- 18 

4 

4 

4 

4 

NO  CHANGE 

EXCEL/EXCEL 

ACC-24 

4 

4 

4 

4 

NO  CHANGE 

EXCEL/EXCEL 

ACC-28 

4 

4 

1 

1 

DECREASE 

EXCEL /UNSAT 

ACC-31 

3 

3 

3 

3 

NO  CHANGE 

SAT /SAT 

ACC-32 

4 

4 

4 

4 

NO  CHANGE 

EXCEL/EXCEL 

ACC-33 

3 

3 

2 

2 

DECREASE 

SAT/MARG 

AMC-1 

1 

1 

1 

1 

NO  CHANGE 

UNSAT/UNSAT 

ANC-9 

4 

4 

1 

1 

DECREASE 

EXCEL/UNSAT 

AMC-10 

1 

1 

4 

4 

INCREASE 

UNSAT/ EXCEL 

Like  the  tabular  analysis  of  ECAMP  Effectiveness  Model 
data  using  standardized  ECAMP  findings  previously  presented 
in  Table  14,  the  analysis  using  unstandardized  findings 
which  is  shown  in  Table  15  identifies  the  same  eleven 
installations  as  having  changes  in  ECAMP  effectiveness. 
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This  similarity  exists  because  the  effectiveness  trends  are 
controlled  by  the  number  of  NOVs  received  by  the  respective 
installations  over  the  two  year  period  which  was  constant. 
However,  the  overall  effectiveness  ratings  are  influenced  by 
the  number  of  standardized  or  unstandardized  ECAMP  findings; 
thus,  these  ratings  are  variable.  To  aid  in  identifying  the 
effects  of  standardizing  ECAMP  findings  on  overall 
effectiveness  ratings.  Table  16  lists  base  specific 
standardization  factors  and  the  overall  effectiveness 
ratings  for  both  analyses,  indicates  whether  unstandardized 
findings  resulted  in  better  or  worse  ratings,  and  identifies 
the  number  of  times  (1-4)  when  the  data  points  in  Figures 
18-21  fell  in  different  or  the  same  effectiveness  quadrant 
as  those  in  corresponding  Figures  22-25. 
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TABLE  16 


OVERALL  ECAMP  EFFECTIVENESS  RATING  COMPARISON 


BASS 

BEE 

STD 

FACTOR 

OVERALL 

RATINGS 

STD 

FINDINGS 

OVERALL 

RATINGS 

UNSTD 

FINDINGS 

UNSTD 

RATING 

BETTER 

OR  WORSE 

FIGURE 

QUADRANT 

DIFFERENT/ 

SAME 

AFMC-1 

9 

MARG/MARG 

UNSAT/ UNSAT 

WORSE 

4/0 

AFMC-3 

35 

EXCEL/EXCEL 

EXCEL/EXCEL 

SAME 

0/4 

AFMC-4 

51 

UNSAT/ UNSAT 

MARG/MARG 

BETTER 

4/0 

AFMC-5 

16 

EXCEL/UNSAT 

SAT/MARG 

WOR/BET 

2/2 

AFNC-6 

43 

EXCEL/EXCEL 

SAT/ SAT 

WORSE 

4/0 

AFMC-7 

32 

EXCEL/UNSAT 

SAT/MARG 

WOR/BET 

4/0 

AFHC-9 

47 

UNSAT/UNSAT 

MARG/MARG 

BETTER 

2/2 

AFMC-10 

24 

UNSAT /UNSAT 

MARG/MARG 

BETTER 

4/0 

AFMC-1 2 

D 

EXCEL /UNSAT 

EXCEL/UNSAT 

SAME 

0/4 

ACC- 3 

12 

UNSAT /UNSAT 

UNSAT/UNSAT 

SAME 

0/4 

ACC- 4 

12 

UNSAT /EXCEL 

UNSAT/EXCEL 

SAME 

0/4 

ACC- 5 

5 

MARG/EXCEL 

UNSAT/EXCEL 

WORSE 

2/2 

ACC-6 

8 

EXCEL/UNSAT 

EXCEL/UNSAT 

SAME 

0/4 

ACC- 8 

9 

MARG/SAT 

UNSAT/EXCEL 

WOR/BET 

2/2 

ACC- 10 

5 

EXCEL/EXCEL 

EXCEL/EXCEL 

SAME 

0/4 

TABLE  16 


OVERALL  ECAMP  EFFECTIVENESS  RATING  COMPARISON 


BASE 

BEE 

STD 

FACTOR 

OVERALL 

RATINGS 

STD 

FINDINGS 

OVERALL 

RATINGS 

UNSTD 

FINDINGS 

UNSTD 

RATING 

BETTER 

OR  WORSE 

FIGURE 

QUADRANT 

DIFFERENT/ 

SANE 

ACC- 18 

6 

SAT/SAT 

EXCEL/EXCEL 

BETTER 

4/0 

ACC-24 

9 

EXCEL/EXCEL 

EXCEL/EXCEL 

SAME 

0/4 

ACC-28 

9 

EXCEL/ UNSAT 

EXCEL/UNSAT 

SAME 

0/4 

ACC-31 

9 

SAT/ SAT 

SAT/ SAT 

SAME 

0/4 

ACC-32 

8 

EXCEL/EXCEL 

EXCEL/ EXCEL 

SAME 

0/4 

ACC-33 

11 

SAT/MARG 

SAT/MARG 

SAME 

0/4 

AMC-1 

13 

UNSAT/UNSAT 

UNSAT /UNSAT 

SAME 

0/4 

AMC-9 

7 

EXCEL/UNSAT 

EXCEL/UNSAT 

SAME 

0/4 

AMC-10 

13 

UNSAT/EXCEL 

UNSAT/EXCEL 

SAME 

0/4 

Table  16  shows  that  32  of  96  data  points  were  in 
different  effectiveness  quadrants  when  unstandardized  ECAMP 
findings  were  used  in  the  model  as  compared  to  standardized 
findings.  These  quadrant  shifts  resulted  in  10  of  24  bases 
having  different  overall  ECAMP  effectiveness  ratings.  Of 
the  ten  installations  with  different  overall  ratings  when 
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unstandardized  findings  were  used,  four  had  better  ratings, 
three  had  worse,  and  three  had  worse  ratings  using  NOVs  1- 
year  prior  to  the  ECAMP  evaluation  and  better  ratings  using 
NOVs  from  1-year  after  the  ECAMP  (WOR/BET) .  Further 
inspection  of  Table  16  also  reveals  that  seven  of  nine  AFMC 
bases  in  the  study  had  different  overall  effectiveness 
ratings  when  standardized  findings  were  replaced  with 
unstandardized  findings.  In  contrast,  only  three  of  twelve 
and  zero  of  three  ACC  and  AMC  study  bases  had  different 
overall  effectiveness  ratings.  In  addition,  the  average 
value  of  the  BEE  standardization  factor  for  the  ten  bases 
exhibiting  different  effectiveness  ratings  was  24.2  while 
the  average  standardization  factor  for  the  14  bases  with  the 
same  effectiveness  rating  was  14.42.  Although  inconclusive, 
these  results  suggest  that  the  BEE  standardization  factor 
does  account  for  the  dissimilarities  between  large  and  small 
installations  as  expected  and  desired. 

Overall,  reanalyzing  the  NOV  and  ECAMP  data  for  the 
same  24  bases  using  unstandardized  findings  demonstrated  the 
flexibility  of  the  proposed  model  and  provided  additional 
insight  into  the  effects  and  potential  benefits  of 
standardizing  ECAMP  findings  using  BEE  manpower 
authorizations.  Ultimately,  by  using  the  proposed  ECAMP 
Effectiveness  Model,  AF  decision-makers  can  quickly  and 
easily  identify  both  installations  with  effective  ECAMP 
programs  and  those  bases  where  ECAMP  programs  may  need  more 
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attention  or  resources  to  prevent  future  environmental 
problems  and  improve  environmental  compliance. 

Summary.  Theoretically,  an  effective  ECAMP  should  be 
characterized  by  a  negative  correlation  between  NOVs  and 
standardized  ECAMP  findings.  Hypothetically,  an  effective 
ECAMP  would  be  expected  to  progress  through  four  distinct 
phases  culminating  in  a  perfectly  effective  auditing  program 
where  all  potential  compliance  problems  are  identified, 
corrected  and  subsequently  prevented. 

Realistically,  ECAMP  effectiveness  like  environmental 
compliance  is  constantly  changing  due  to  confounding  factors 
such  as  manpower  shortages,  changing  environmental 
regulations,  and  varying  levels  of  regulatory  oversight. 
Thus,  AF  decision-makers  need  a  quick,  easy  method  of 
measuring  relative  ECAMP  effectiveness  so  that  scarce 
resources  can  be  properly  used  to  improve  ECAMP 
effectiveness  and  prevent  NOVs.  By  applying  the  proposed 
ECAMP  Effectiveness  Model  to  24  bases  with  recent  ECAMP 
evaluations  and  NOV  data  for  both  1-year  prior  to  and  1-year 
after  the  evaluation,  this  research  demonstrated  that  the 
proposed  model  provides  a  quick  and  easy,  visual  indication 
of  an  installation's  relative  ECAMP  effectiveness. 

Additional  analyses  using  unstandardized  ECAMP  findings  also 
demonstrated  the  flexibility  of  the  proposed  model  and 
suggested  that  the  BEE  standardization  factor  does  account 
for  the  dissimilarities  between  large  and  small 
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installations  as  expected  and  desired.  This  information  can 
then  be  used  to  assist  AF  decision-makers  in  identifying  the 
installations  that  require  resources  to  prevent  future 
environmental  problems  and  improve  environmental  compliance. 

If  approved  and  implemented  by  the  AF,  the  proposed 
ECAMP  Effectiveness  Model  could  assist  decision-makers  in 
properly  allocating  and  apportioning  scarce  resources  among 
installations  with  competing  environmental  compliance 
requirements.  Ultimately,  using  the  proposed  model  to 
identify  installations  where  resources  are  needed  ^aould 
improve  ECAMP  effectiveness  thereby  reducing  AF  NOVs  and 
preventing  fines  and  penalties  pursuant  to  the  Federal 
Facilities  Compliance  Act. 
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V.  Conclusions  and  Recommendations 


Overview 

Chapter  V  reviews  the  research  problem  and  objectives, 
summarizes  and  describes  how  the  research  results  achieved 
the  three  research  objectives,  concludes  that  the  ECAMP 
Effectiveness  Model  developed  in  this  research  provides  a 
quick  and  easy,  visual  indication  of  an  installation's 
relative  ECAMP  effectiveness,  and  recommends  that  AF  and 
MAJCOM  environmental  leaders  review  the  research  results  and 
consider  using  the  proposed  ECAMP  Effectiveness  Model  as  a 
metric  to  measure  ECAMP  effectiveness. 

Problem  Review 

The  DoD  and  AF  are  committed  to  being  national  leaders 
in  protecting  and  enhancing  the  environment  and  achieving 
environmental  compliance  (38:v;  39:1).  This  commitment  is 
exemplified  by  the  AF  Chief  of  Staff's  goal  of  "no  notices 
of  violation”  (36) .  However,  the  number  of  AF  NOVs  from 
1990-1992  rose  from  103  to  178  an  increase  of  73  percent  (7; 
13;  25).  A  July  1992  AF  Inspector  General  Report  also 
concluded  that  ECAMP  findings  were  not  being  fully  utilized 
and  that  "several  commands  were  not  effectively  using 
external  ECAMP  reports  to  direct  corrective  actions  or 
allocate  resources"  (4:6).  To  aid  AF  leaders  in  solving 
these  problems,  this  research  developed  an  ECAMP 
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Effectiveness  Model  that  uses  ECAMP  findings  and  historical 
NOV  data  to  assist  decision-makers  in  identifying  the 
installations  that  require  resource  allocation  to  prevent 
future  environmental  problems  and  improve  environmental 
compliance. 

Results  Summary 

The  proposed  ECAMP  Effectiveness  Model  was  developed 
from  the  results  of  a  research  methodology  that  achieved  the 
following  research  objectives. 

1.  Demonstrated  that  environmentally-related  manpower 
determinants  can  be  used  to  standardize  ECAMP  findings 
thereby  enabling  unbiased  comparisons  between  dissimilar  AF 
installations. 

2.  Developed  a  methodology  that  uses  the  standardized 
ECAMP  findings  to  assist  AF  decision-makers  in  identifying 
the  installations  that  require  resource  allocation  to 
prevent  future  environmental  problems  and  improve 
environmental  compliance. 

3.  Tested,  evaluated,  and  revised  the  methodology  for 
possible  use  throughout  the  AF. 

Objective  1.  To  accomplish  research  objective  1,  an 
extensive  literature  review  was  performed.  This  review 
concluded  that  using  environmentally-related  manpower 
determinants  to  standardize  ECAMP  findings  for  respective 
installations  where  they  are  being  applied  should  allow 
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unbiased  comparisons  of  ECAMP  findings  from  dissimilar 
installations  using  these  same  manpower  determinants-  Based 
on  the  results  of  the  literature  review,  it  was  also 
concluded  that  the  BEE  manpower  determinant  should  be  the 
best  standardizing  factor  since  it  has  matured  over  time 
with  the  developing  environmental  legislation.  This 
conclusion  was  also  supported  by  subsequent  manpower 
regression  analyses  which  suggested  that  approximately  82 
percent  of  the  variations  in  environmental  manpower 
authorizations  (Y)  could  be  explained  by  the  variations  in 
BEE  manpower  authorizations  (X)  . 

Although  the  validity  of  using  BEE  manpower 
authorizations  could  not  be  verified  directly,  seven  of  ten 
r^  values  were  higher  with  standardized  findings  than  with 
unstandardized  findings  in  the  expanded  study.  In  addition, 
a  comparative  analysis  of  ECAMP  Effectiveness  Model  results 
using  both  standardized  and  unstandardized  ECAMP  findings 
revealed  that  the  average  value  of  the  BEE  standardization 
factor  for  the  ten  bases  exhibiting  different  overall 
effectiveness  rating  was  24.2  while  the  average 
standardization  factor  for  the  14  bases  with  the  same 
overall  effectiveness  ratings  was  14.42.  Although 
inconclusive,  these  results  suggest  that  the  BEE 
standardization  factor  does  account  for  the  dissimilarities 
between  large  and  small  installations  as  desired  and 
expected.  Moreover,  weighting  ECAMP  findings  before  making 
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resource  allocation  decisions  using  a  standardization  factor 
such  as  BEE  manpower  authorizations  that  should  account  for 
the  installation  specific  environmental  compliance 
requirements  makes  good  managerial  sense. 

Objective  2.  The  first  step  in  achieving  research 
objective  2  was  to  test  the  theory  that  there  should  be  a 
direct  relationship  between  standardized  ECAMP  findings  and 
environmental  compliance  by  performing  regression  analyses 
using  standardized  ECAMP  findings  and  historical  NOV  data 
from  nine  AFMC  bases.  Although  the  majority  of  the  pilot 
study  coefficients  of  determination  were  very  low,  the  0.79 
and  0.84  r^  values  for  five  former  AFLC  installations  were 
potentially  significant  and  clearly  justified  expanding  this 
study  to  installations  in  other  commands. 

In  contrast  to  the  pilot  study,  no  significant 
relationships  between  NOVS  and  standardized  or 
unstandardized  ECAMP  findings  were  identified  in  the 
expanded  study.  However,  a  detailed  review  of  the  expanded 
study  data  did  result  in  a  theoretical  framework  for  the 
proposed  ECAMP  Effectiveness  Model.  The  proposed  model  is 
patterned  after  the  two-factor  theory  of  leadership,  and  the 
model  is  based  on  the  premise  that  fewer  NOVs  and  fewer 
ECAMP  findings  are  best. 

Objective  3.  Research  objective  3  was  achieved  by 
testing  the  ECAMP  Effectiveness  Model  both  prospectively  and 
retrospectively.  This  was  accomplished  by  using  the 
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proposed  model  to  analyze  historical  NOV  data  and  both 
standardized  and  unstandardized  ECAMP  findings  from  24  bases 
with  ECAMP  evaluations  in  1991  or  1992  and  with  NOV  data  for 
both  1-year  prior  to  and  l-year  after  the  evaluation.  The 
model  results  were  then  evaluated  and  compared  using  a 
tabular  format  based  on  the  standard  AF  rating  system  of 
unsatisfactory,  marginal,  satisfactory,  and  excellent. 

Based  on  this  testing  and  analysis,  the  proposed  ECAMP 
Effectiveness  Model  should  allow  AF  decision-makers  to 
quickly  and  easily  identify  installations  with  effective 
ECAMP  programs  and  those  bases  where  ECAMP  programs  need 
more  attention  and/or  resources. 

Conclusions 

Using  the  ECAMP  Effectiveness  Model  developed  in  this 
research  provides  a  quick  and  easy,  visual  indication  of  an 
installation's  relative  ECAMP  effectiveness.  The  proposed 
model  is  extremely  flexible;  it  can  be  easily  adjusted  to 
reflect  changing  AF  or  command  goals  or  averages;  it  can  be 
used  both  retrospectively  and  prospectively  with  either 
standardized  and/or  unstandardized  ECAMP  findings;  and  it 
uses  manpower,  NOV,  and  ECAMP  report  data  that  is  readily 
available  to  decision-makers.  If  approved  and  implemented 
by  the  AF,  the  proposed  ECAMP  Effectiveness  Model  should 
assist  decision-makers  in  properly  allocating  and 
apportioning  scarce  resources  among  installations  with 
competing  environmental  compliance  requirements.  The 
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proposed  model  provides  decision-makers  with  an  easy  to  use 
management  tool  that  can  help  them  to  identify  the 
installations  that  may  need  resources.  Ultimately,  using 
the  ECAMP  Effectiveness  Model  should  improve  ECAMP 
effectiveness  thereby  reducing  AF  NOVs  and  preventing  fines 
and  penalties  pursuant  to  the  Federal  Facilities  Compliance 
Act. 

Recommendations 

Air  Force  and  MAJCOM  environmental  leaders  should 
review  the  results  of  this  research  and  adopt  the  proposed 
ECAMP  Effectiveness  Model  as  a  metric  to  measure  ECAMP 
effectiveness  and  assist  AF  decision-makers  in  allocating 
scarce  resources  to  prevent  future  environmental  problems 
and  improve  environmental  compliance.  Any  questions, 
problems,  or  concerns  with  the  proposed  model  should  be 
documented  and  forwarded  to  the  Air  Force  Institute  of 
Technology's  Department  of  Environmental  Engineering  and 
Management  for  possible  model  revision  and/or  follow-on 
research. 
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ADoendix  A:  Manoower  Rearer ion  Analysis  Data 


(l;  9;  14;  32; 

33;  37) 

AUTHORIZED 

AUTHORIZED 

BASE 

BEE  MANPOWER 

ENV.  MANPOWER 

AFMC-1 

9 

9 

AFMC-2 

31 

80 

AFNC-3 

44 

47 

AFMC-4 

51 

56 

AFMC-5 

16 

19 

AFMC-6 

43 

82 

APMC-7 

32 

90 

AFMC'-8 

6 

10 

AFMC-9 

47 

77 

AFMC-10 

27 

54 

AFMC-11 

5 

2 

AFMC-12 

51 

86 

ACC-1 

6 

NOT  AVAILABLE  CLOSED 

ACC-2 

8 

8 

ACC- 3 

12 

9 

ACC- 4 

12 

10 

ACC- 5 

5 

NOT  AVAILABLE  CLOSING 

ACC-6 

8 

...  . 

9 

125 


126 


ACC-26 


ACC-27 


ACC-28 


ACC-29 


ACC-30 


ACC-31 


ACC-32 


ACC-33 


ACC-34 


AMC-1 


Al.C-2 


AMC-3 


ANC-4 


AMC-8 


AMC-9 


AMC-10 


AUTHORIZED 

AUTHORIZED 

BEE  MANPOWER 

BNV.  MANPOWER 

9 


8 


9 


10 


6 


6 


9 


19 


8 


12-16 


11-10 


13-11 


12-12 


AMC-5 

11 

12-13 

AMC-6 

16 

12-12 

AMC-7 

5 

NOT  AVAILABLE  CLOSED 

10-10 


8-7 


10-9 
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AUTHORIZED 

AUTHORIZED 

BASE 

BEE  MANPOWER 

ENV.  MANPOWER 

AMC-11 

9 

15-12 

AMC-12 

15 

14-17 

AMC-13 

12 

11-12 
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Appendix  B;  AFMC  Specific  Manpower  Regression  Analysis  Data 

(37) 


AUTHORIZED 

AUTHORIZED 

AUTHORIZED 

ACTUAL 

BASE 

AFMC  BEE 

AFMC  EM 

AFMC  EM 

AFMC  EM 

MANPOWER 

MANPOWER 

MANPOWER 

MANPOWER 

APRIL  93 

APRIL  92 

APRIL  93 

APRIL  93 

AFMC-l 

9 

7 

9 

6 

AFMC- 2 

31 

80 

80 

50 

AFMC- 3 

44 

35 

47 

53 

AFMC- 4 

51 

43 

56 

44 

AFMC- 5 

16 

11 

19 

23 

AFMC-6 

43 

65 

82 

83 

AFMC- 7 

32 

72 

90 

90 

AFMC- 8 

6 

11 

10 

8 

AFMC- 9 

47 

79 

77 

70 

AFMC- 10 

27 

51 

54 

26 

AFMC- 11 

5 

2 

2 

2 

AFMC-12 

51 

60 

86 

84 
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ADoendlx  C; 

Exoanded  Studv  ECAMP  Findina  and  NOV  Data 

(7;  9;  13;  25; 

r  29;  42) 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

MAJOR 

TOTAL 

NOVs 

BEE 

BEE 

BEE 

BASE/ 

ECAMP 

ECAMP 

AFTER 

STD 

STD 

STD 

ECAHP  YEAR 

FINDINGS 

FINDINGS 

l-YR 

FACTOR 

MAJOR 

TOTAL 

AFMC-1/92 

47 

82 

3 

9 

5.22 

9.11 

AFMC-3/91 

24 

56 

1 

44 

0.55 

1.27 

AFMC-4/91 

68 

101 

5 

51 

1.33 

1.98 

AFMC-5/91 

54 

74 

2 

16 

3.38 

4.63 

AFMC-6/91 

55 

113 

0 

43 

1.28 

2.63 

AFMC-7/92 

67 

85 

7 

32 

2.09 

2.66 

ArMC-9/92 

44 

88 

5 

47 

0.94 

1.87 

AFNC-10/92 

119 

156 

3 

27 

4.41 

5.78 

AFMC-12/92 

48 

75 

4 

51 

0.94 

1.47 

ACC- 1/90 

19 

34 

5 

6 

3.17 

5.67 

ACC-3/ 91 

42 

58 

5 

12 

3.50 

4.83 

ACC-4/ 91 

43 

73 

1 

12 

3.58 

6.08 

ACC-5/ 91 

25 

44 

0 

5 

5.00 

8.80 

ACC-6/91 

32 

56 

3 

8 

4.00 

7.00 

ACC-8/92 

48 

94 

0 

9 

5.33 

10.44 

ACC-10/91 

22 

54 

0 

5 

4.40 

10.80 
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BASE/ 

ECAMP  YEAR 


ACC-ll/89 


ACC- 16/ 90 


ACC-18/91 


ACC-2 1/90 


ACC-22/89 


ACC-23/89 


ACC-24/92 


ACC-2b/90 


ACC-28/ 91 


ACC-29/90 


ACC-31/89 


ACC-31/92 


ACC-32/91 


ACC-33/92 


ACC-34/89 


(2) 

MAJOR 


FINDINGS 


(3) 

TOTAL 

ECAMP 

FINDINGS 


(4) 

NOVs 

AFTER 

1-YR 


(6) 

(2) 

BEE 

STD 

MAJOR 

STD 

TOTAL 

1.67 

4.83 

2.29 

8.14 

7.00 

12.83 

1.00 

3.57 

1.80 

5.60 

2.75 

5.83 

3.56 

6.67 

2.00 

4.40 

2.00 

6.44 

2.13 

4.38 

1.56 

4.89 

5.89 

10.22 

4.63 

8.50 

5.73 

10.55 

1.29 

2.71 
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BASE/ 


(2) 

NATOR 

SCAMP 


(3) 

TOTAL 


SCAMP  YEAR  FINDINGS  FINDINGS 


(4) 

NOVs 

AFTER 

l-YR 


AMC-1/9I 

17 

34 

AMC-9/92 

12 

32 

AMC- 10/91 

13 

26 

AMC-13/90 

20 

40 

<«) 

(7) 

BEE 

BEE 

STD 

MAJOR 

STD 

TOTAL 

1.31 

2.62 

1.71 

4.57 

1.18 

2.36 

1.67 

3.33 
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I 


Aopendix  D:  ECAMP  Effectiveness  Model  Historical  NOV  Data 


(7;  13;  25) 


(1) 

<2) 

(3) 

MOVE 

(4) 

MOVE 

(5) 

(4) 

(7) 

(8) 

ECAMP 

l-YR 

AFTER 

NOVs 

NOVs 

NOVs 

NOVs 

BASE 

DATE 

PRIOR 

l-YR 

1989 

1990 

1991 

1992 

AFNC-1 

MAR  92 

AFMC-2 

SEP  92 

AFMC-3 

JUL  91 

AFMC>4 

OCT  91 

AFMC-5 

OCT  91 

AFMC-6 

DEC  91 

AFMC-7 

FEB  92 

AFMC-8 

JUL  92 

AFMC-9 

APR  92 

AFNC-10 

JAN  92 

AFNC-11 

MAY  92 

AFMC-12 

JAN  92 

ACC-1 

NOV  90 

ACC- 2 

NONE 

ACC- 3 

AUG  91 

AUG  91 

133 


(1) 

(2) 

(3) 

NOVs 

(4) 

NOVs 

(5) 

<6) 

(7) 

(«) 

BASE 

ECAMP 

1-YR 

AFTER 

1989 

1990 

1991 

1992 

DATE 

PRIOR 

l-YR 

NOVs 

NOVs 

NOVs 

NOVs 

ACC- 5 

JUN  91 

ACC- 6 

AUG  91 

ACC- 6 

FEB  93 

ACC- 7 

NOV  92 

ACC- 8 

MAR  92 

ACC-9 

NOV  92 

ACC- 10 

JUN  91 

ACC- 11 

JUL  89 

ACC- 11 

FEB  93 

ACC- 12 

NONE 

ACC- 13 

NONE 

ACC- I 4 

FEB  93 

ACC-15 

NONE 

ACC- 16 

APR  90 

ACC-16 

JAN  93 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

NOVs 

NOVs 

BASE 

ECAMP 

l-YR 

AFTER 

1989 

1990 

1991 

1992  1 

DATE 

PRIOR 

l-YR 

NOVs 

NOVs 

NOVs 

NOVs  1 

ACC- 17 

OCT  92 

ACC- 18 

OCT  91 

ACr-19 

JUN  92 

ACC-20 

OCT  92 

ACC-21 

NOV  90 

ACC-22 

DEC  89 

ACC-22 

JUL  92 

ACC-23 

AUG  89 

ACC-23 

DEC  92 

ACC-24 

FEB  92 

ACC-25 

JUN  90 

ACC-25 

OCT  92 

ACC-26 

AUG  92 

ACC-27 

JAN  93 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

NOVs 

NOVs 

SCAMP 

1-YR 

AFTER 

1989 

1990 

1991 

1992 

BASE 

DATE 

PRIOR 

l-YR 

NOVs 

NOVs 

NOVs 

NOVs 

MAR 

91 

AUG 

90 

JUL 

92 

MAR 

93 

JUN 

89 

MAR 

92 

DEC 

91 

FEB 

92 

AUG 

89 

FEB 

93 

MAR 

91 

ANC-2  NONE  N/A 


AMC-3  NONE  N/A 


AMC-4 


SEP  92 


N/A 


0 


(1) 

(2) 

(3) 

(4) 

(5) 

(«) 

(7) 

<8) 

NOVs 

NOVs 

BCAMP 

1-YR 

AFTER 

NOVs 

NOVs 

NOVs 

NOVs 

BASE 

DATE 

PRIOR 

1-YR 

1989 

1990 

1991 

1992 

AMC-5 

NONE 

N/A 

N/A 

0 

1 

0 

1 

AMC-6 

NONE 

N/A 

N/A 

0 

2 

1 

1 

AMC-7 

SEP  92 

2 

N/A 

0 

0 

2 

2 

AMC-8 

NONE 

N/A 

N/A 

0 

1 

0 

3 

AMC-9 

APR  92 

1 

2 

0 

0 

1 

2 

AMC-10 

SEP  91 

2 

0 

0 

1 

2 

0 

AMC-11 

AUG  92 

4 

N/A 

0 

3 

8 

0 

AMC-12 

NONE 

N/A 

N/A 

0 

1 

1 

3 

AMC-13 

DEC  90 

0 

1 

0 

0 

1 

1 
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Appendix  E;  Effectiveness  Model  Historical  ECAMP  Findings 

(9;  29;  42) 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

MAJOR 

TOTAL 

BEE 

BASE 

ECAMP 

ECAMP 

SCAMP 

STD 

STD 

STD 

DATE 

FINDINGS 

FINDINGS 

FACTOR 

MAJOR 

TOTAL 

AFMC-1 

MAR  92 

47 

82 

9 

5.22 

9.11 

AFMC-2 

SEP  92 

198 

253 

31 

6.39 

8.16 

AFMC-3 

JUL  91 

24 

56 

44 

0.55 

1.27 

AFMC-4 

OCT  91 

68 

101 

51 

1.33 

1.98 

AFMC-5 

OCT  91 

54 

74 

16 

3.38 

4.63 

AFMC-6 

DEC  91 

55 

113 

43 

1.28 

2.63 

AFMC-7 

FEB  92 

67 

85 

32 

2.09 

2.66 

AFNC-8 

JUL  92 

62 

100 

6 

10.33 

16.67 

AFMC-9 

APR  92 

44 

88 

47 

0.94 

1.87 

AFMC-10 

JAN  92 

119 

156 

27 

4.41 

5.78 

AFMC-1 1 

MAY  92 

29 

71 

5 

5.80 

14.20 

AFMC-12 

JAN  92 

48 

75 

51 

0.94 

1.47 

ACC-1 

NOV  90 

19 

34 

6 

3.17 

5.67 

ACC- 2 

NOT  AVAIL 

N/A 

N/A 

N/A 

N/A?? 

N/A?? 

ACC- 3 

AUG  91 

42 

58 

12 

3.50 

4.83 

ACC- 4 

AUG  91 

43 

73 

12 

3.58 

6.08 
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(3) 

MAJOR 


ECAMP 

DATE 


(4) 

TOTAL 

ECAMP 


ACC- 5 

JUN  91 

ACC- 6 

AUG  91 

ACC-6 

FEB  93 

ACC-7 

NOV  92 

ACC-8 

MAR  92 

ACC- 9 

NOV  92 

ACC- 10 

JUN  91 

ACC-11 

JUL  89 

ACC- 11 

FEB  93 

ACC- 12 

NOT  AVAIL 

ACC-13 

NOT  AVAIL 

ACC- 14 

FEB  93 

ACC- 15 

NO*  AVAIL 

ACC- 16 

APR  90 

ACC- 16 

JAN  93 

(6) 

(7) 

BEE 

STD 

MAJOR 

STD  1 
TOTAL 

5.00 

8.80 

4.00 

7.00 

8.63 

19.88 

9.88 

20.13 

5.33 

10.44 

10.86 

16.29 

4.40 

10.80 

1.67 

4.83 

9.17 

17.50 

N/A 

N/A 

N/A 

N/A 

5.10 

10.30 

N/A 

N/A 

2.29 

8.14 

7.00 

12.14 
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ACC- 17 


ACC-18 


ACC- 19 


ACC- 20 


ACC-21 


ACC-22 


ACC-22 


ACC-23 


ECAMP 

DATE 


OCT  92 


OCT  91 


JUN  92 


OCT  92 


NOV  90 


DEC  89 


JUL  92 


AUG  89 


(3) 

HAJOR 

<4) 

TOTAL 

ECAMP 

ECAMP 

FINDINGS 

FINDINGS 

14.86 


12.83 


13.50 


12.33 


8.43 


7.00 


8.33 


7.67 


1.00  3.57 


10  1.80  5.6 


10  4.00  8.10 


5  5.83 


ACC-23 
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endix  G:  Effectiveness  Model  Data  Figures  22-25 
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